Paging datafeed

@bloxwap/chart/datafeed replaces a TradingView UDF datafeed. You write one function that reads bars for a time window. The datafeed loads recent history, pages older history in as the user scrolls left, and folds live price ticks into the forming bar. Superseded reads are aborted and never reach the chart.

Create a chart with a datafeedCopy Anchor Link

import { createDatafeedChart } from '@bloxwap/chart/datafeed';

const { chart, datafeed, destroy } = createDatafeedChart({
  container: canvas,          // any createChart option works here
  autoResize: true,
  preset: 'bloxwapDark',
  // Bars opening within [fromMs, toMs], both inclusive. Hand `signal` to fetch.
  fetchBars: ({ symbol, intervalMs, fromMs, toMs, signal }) => readCandles(symbol, intervalMs, fromMs, toMs, signal),
  symbol: 'BTC',
  intervalMs: 15 * 60_000,
  onError: (error) => console.warn('chart data', error),
});

priceFeed.subscribe('BTC', (mid) => datafeed.pushTick(mid, 'BTC'));
await datafeed.setSymbol('ETH', 60_000); // switch symbol or interval
datafeed.subscribeState(({ loading, exhausted, error }) => renderSpinner(loading, exhausted, error));

// When the view goes away:
destroy(); // the datafeed first, then the chart

fetchBars receives a FetchBarsRequest and resolves to every candle the source has in the window, in any order. Duplicates and rows outside the window are tolerated. A window never holds more than pageSize buckets.

Candle time stays in UNIX seconds. Only the request window is in milliseconds.

To bind a datafeed to a chart you already have (one from <Chart onReady>, or the one the toolbar wraps), use attachDatafeed(chart, options) with the same options minus the chart ones. datafeed.destroy() then detaches it and leaves the chart alive.

OptionsCopy Anchor Link

initialBars, pageBars, pageSize, and maxEmptyPages must be positive integers, and lazyLoadThreshold must be finite and nonnegative. Invalid values throw when the datafeed is created.

How loading worksCopy Anchor Link

• Initial load: setSymbol(symbol, intervalMs) aborts every read of the previous symbol, loads initialBars ending now, and shows them scrolled to the latest bar. It also sets timeAxis.intervalMs and timeScale.intervalMs, so the bar-close countdown and a continuous time axis use the right interval. Live folding restarts from the last bar.
• Switching: results that belong to a previous symbol never reach the chart, even if your fetchBars ignores the signal. If a switch fails, the chart is cleared so the old symbol's bars never show under the new name.
• Lazy history: when fewer than lazyLoadThreshold bars hide past the left edge, the datafeed loads pageBars older bars and inserts them with chart.prependData. The view stays on the same bars. Only one read runs at a time, and the edge is checked again after each page, so a zoomed-out view keeps filling itself.
• Paging: each request covers at most pageSize bars. The next request ends 1 ms before the earliest bar received, and rows outside the requested window are dropped, so no bucket is read twice or skipped. A page that comes back short (the source has no bar for some buckets) doesn't end the read: paging goes on until the load has its bars.
• Holes: an empty page is taken as a hole in the source, such as a weekend, a closed session, or an outage. The next request ends where the empty one began and spans a whole pageSize, so the hole is stepped over in as few reads as possible. An initial load made while a market is closed steps back to its last session the same way.
• End of history: after maxEmptyPages empty pages in a row, exhausted becomes true and no more requests are made. Reaching the start of a symbol's history therefore costs that many empty reads, once. An initial load that reaches it is exhausted right away.
• Errors: failures appear in datafeed.error and go to onError. There is no retry loop. The next approach to the left edge retries once a backoff has passed: RETRY_BACKOFF_MS (1 s) after a failure, doubling with each failure in a row up to MAX_RETRY_BACKOFF_MS (30 s), so dragging at the edge doesn't hammer a rate-limited source. A loadMore() call retries at once, and a successful load resets the backoff. A failed initial load is retried by loadMore().
• Timers: the datafeed starts none. Loading is driven by viewport changes and your calls, so a backoff that ends retries at the next viewport change after it.

loadMore(bars?) loads up to bars (default pageBars) older bars and resolves to the number added. It never rejects. While a read is in flight it returns that read.

Live barsCopy Anchor Link

priceFeed.subscribe(symbol, (mid) => datafeed.pushTick(mid, symbol));

pushTick(price, symbol?) updates the forming bar, opens new buckets at the previous close, and backfills missed buckets:

• Buckets are epoch-aligned multiples of intervalMs.
• A tick in the held bucket updates high, low, and close. The next bucket opens at the previous close with volume 0.
• A tick that lands a whole bucket late, or after 8 seconds of silence (2 seconds when it opens a bucket), triggers a gap repair. The repair reads history from the held bar through now with fetchGap, or fetchBars when you gave no fetchGap. Ticks that arrive meanwhile are buffered and replayed afterwards, even when the read fails.
• A repair's burst of bars renders once.

Ticks carry prices only, so a bar formed from ticks has the volume of its last history read (0 for a bucket opened by a tick) until a repair reads it again. This matches the app's LiveBarFolder.

Once per frameCopy Anchor Link

A busy feed sends many ticks per frame, and each one repaints the chart by default. Pass a scheduler to render them once per frame instead, in the same batch as pointer work, animations, and the countdown:

import { createDatafeedChart } from '@bloxwap/chart/datafeed';
import { createFrameScheduler, startCountdownTicker } from '@bloxwap/chart/ui';

const frames = createFrameScheduler(window, (update) => chart.batch(update));
const { chart, datafeed } = createDatafeedChart({ container: canvas, fetchBars, scheduler: frames });
startCountdownTicker({ chart, window, scheduler: frames });
priceFeed.subscribe('BTC', (mid) => datafeed.pushTick(mid, 'BTC')); // no queue of your own

• pushTick still folds each tick at once, so every tick lands in the bucket of the moment it arrives.
• The next frame applies what changed with appendData, in one render: every bar that closed since the last frame, in order, then the forming one. Only the latest state of each bar is applied.
• A hidden tab gets no animation frames. Folding goes on meanwhile, and the chart catches up in one render when the tab is shown.
• setSymbol and destroy drop bars not yet applied and cancel their frame.
• The chart delivers its events when the frame's chart.batch ends, after the datafeed's part of the frame. A chart listener that throws there is thrown from the frame, like any listener error from pointer work in it, and doesn't reach onError. A scheduler that doesn't wrap its frames in chart.batch sends it to onError.

Without a scheduler, each tick's bar is applied at once, as before.

Listener rulesCopy Anchor Link

• pushTick is safe in any price-feed callback. It never throws: errors from publishing a bar (for example, a chart listener that throws) go to onError. With a scheduler the bar is published in a later frame; see once per frame for where a listener's error goes then.
• Ticks are ignored until history has landed, and ticks for another symbol are ignored when you pass symbol. Non-finite and non-positive prices are ignored too. Subscribe to the price feed whenever you like; you don't need to wait for setSymbol.
• subscribeState(listener) calls listener with a DatafeedState snapshot (symbol, intervalMs, loading, exhausted, error) when a load starts and when it settles. It does not call the listener on subscribe; read datafeed.state for the current value. A listener that throws is reported to onError, and the other listeners still run. It returns an unsubscribe function, and destroy() removes every state listener.
• The datafeed owns the chart's data while it is attached. Don't call setData or appendData for the datafeed's symbol yourself. Use setSymbol to reload and pushTick for live prices.
• Chart events keep working. Every datafeed load goes through setData, prependData, or appendData, so subscribeDataLoad reports 'set', 'prepend', 'append', and 'update' loads as usual.
• Destroy it. Call destroy() from createDatafeedChart, or datafeed.destroy() before destroying the chart yourself. Otherwise reads already in flight still write into the destroyed chart's store. This is harmless, since rendering is a no-op, but wasteful.

Abort controllersCopy Anchor Link

Each setSymbol makes one AbortController. Every history and gap read for that symbol gets its signal, and a switch or destroy() aborts it. By default the controller comes from the global AbortController. Pass createAbortController for a runtime without one, or to observe aborts:

createDatafeedChart({
  container: canvas,
  fetchBars,
  createAbortController: () => new AbortControllerPolyfill(), // a spec-complete polyfill
});

The factory returns an AbortControllerLike: a signal and abort(reason?). The datafeed hands that signal to fetchBars, and usually on to fetch, so it is typed as your project's AbortSignal whenever DOM or Node typings are loaded. Return a real or spec-complete AbortSignal there. A bare { aborted, reason } signal type-checks only in a project without those typings, and only works with a fetchBars that doesn't pass it to fetch. The datafeed itself reads just signal.aborted and signal.reason around every read, so an abort takes effect even when fetchBars ignores the signal. loadHistory takes the same option, which it uses only when you pass no signal.

Hyperliquid exampleCopy Anchor Link

This reads Hyperliquid's public candleSnapshot endpoint and streams mid prices from its allMids WebSocket channel.

import { createDatafeedChart, type FetchBars } from '@bloxwap/chart/datafeed';

const INFO_URL = 'https://api.hyperliquid.xyz/info';
const INTERVALS: Record<number, string> = {
  60_000: '1m', 300_000: '5m', 900_000: '15m', 3_600_000: '1h', 14_400_000: '4h',
  86_400_000: '1d', 259_200_000: '3d', 604_800_000: '1w', 2_592_000_000: '1M',
};

interface HyperliquidCandle { t: number; o: string; h: string; l: string; c: string; v: string }

const fetchBars: FetchBars = async ({ symbol, intervalMs, fromMs, toMs, signal }) => {
  const response = await fetch(INFO_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      type: 'candleSnapshot',
      req: { coin: symbol, interval: INTERVALS[intervalMs], startTime: fromMs, endTime: toMs },
    }),
    signal,
  });
  if (!response.ok) throw new Error(`candleSnapshot failed: ${response.status}`);
  const rows = (await response.json()) as HyperliquidCandle[];
  return rows.map((k) => ({ time: k.t / 1000, open: +k.o, high: +k.h, low: +k.l, close: +k.c, volume: +k.v }));
};

const { chart, datafeed, destroy } = createDatafeedChart({
  container: document.querySelector<HTMLCanvasElement>('#chart')!,
  autoResize: true,
  preset: 'bloxwapDark',
  fetchBars,
  symbol: 'BTC',
  intervalMs: 900_000,
});

const socket = new WebSocket('wss://api.hyperliquid.xyz/ws');
socket.onopen = () => socket.send(JSON.stringify({ method: 'subscribe', subscription: { type: 'allMids' } }));
socket.onmessage = (event) => {
  const message = JSON.parse(event.data);
  const symbol = datafeed.symbol;
  const mid = symbol === null || message.channel !== 'allMids' ? undefined : message.data.mids[symbol];
  if (mid !== undefined) datafeed.pushTick(Number(mid), symbol!);
};

candleSnapshot returns at most 5000 rows per request, which is the default pageSize. Its windows are inclusive at both ends, which is what fetchBars expects.

Building blocksCopy Anchor Link

The pieces behind the datafeed are exported for custom loaders:

createLiveBarFolder returns { pushTick, onPrice, bar, filling, dispose }. Its onError receives errors thrown by onBar (or batch) while a gap repair publishes, which would otherwise be lost in the asynchronous repair.

Known limitsCopy Anchor Link

• The datafeed can't tell a long hole from the start of history. A hole of (maxEmptyPages - 1) × pageSize buckets or more can end paging early: 10,000 buckets with the defaults, about a week of 1m bars. Raise maxEmptyPages for a source with a small pageSize and long closures. Hyperliquid's history is contiguous, so this doesn't happen there.
• Paging takes each page to hold every bar the source has from its earliest row to the end of the window. Keep pageSize at or below your source's per-request row cap: a source that cuts a longer window down to its earliest rows (rather than its latest) would otherwise leave holes that nothing reports.
• A gap read that fails leaves the missed buckets empty; the tick opens its own bucket. The failure is reported through onError.