fix(data): chunk the remaining candle-request windows under the venue cap

[eaitbrahim]

The bug

Coinbase rejects any candle request spanning more than ~350 candles:

400 INVALID_ARGUMENT ... "number of candles requested should be less than 350"

#269 fixed this in market_feed.poll_once. An independent review then found two more sites with the same defect. Both are fixed here, so every candle-request windowing site in the codebase now agrees.

Both are latent. Neither is firing today, and this PR is preventative rather than a repair of live damage. What makes them worth closing is the failure mode: each fails quietly, and one of them fails quietly forever.

Defect 1 — repair.py cannot heal an interior hole larger than the cap

gaps.detect puts no cap on n_missing, so an interior hole can be arbitrarily large. repair_series asked for the whole hole in one call:

fetched = client.get_candles(
    product, granularity, window.start_ts - step, window.end_ts + step
)

A gap of roughly 349+ candles therefore 400s. Because the per-window except swallows the exception into result.errors and continues, the failure is silent and permanent — every scheduled repair retries the same oversized request, and the only symptom is a series that quietly never repairs.

This is not currently triggering

Measured across the deployment (24 products, ONE_HOUR and ONE_DAY), the largest interior gap is:

product	granularity	interior gap
ICP-USD	ONE_HOUR	15 bars
WLD-USD	ONE_HOUR	6
PAXG-USD	ONE_HOUR	6
ZEC-USD	ONE_HOUR	5

Series with an interior gap ≥ 349, i.e. that repair cannot heal: 0. The worst hole in the entire cache is 15 bars against a ~349 limit.

So the justification is "close a latent trap before it bites", not "heal an existing hole". It still earns its place: an interior gap that large is entirely reachable — a multi-day venue outage, a delisting and relisting, or a product added back after a long absence — and a silent, self-perpetuating failure is precisely the kind that goes unnoticed until much later.

Correcting the record: an earlier draft of this PR claimed the 553-candle ZEC-USD figure from the #269 incident was an interior hole repair could never heal. That was wrong. Those 553 candles were end-of-series staleness — a missing tail, which poll_once/ensure_history handle by catching up forward. It was never an interior gap, so repair was never going to be asked to heal it. 553 survives in this PR only as the oversized-gap size used in the tests.

The fix

The widened window is now paged in MAX_CANDLES_PER_REQUEST-sized chunks via a new _fetch_window_chunked helper, importing the constant from keel.data.history rather than declaring a second one — a duplicated venue limit is the same bug class being fixed. Each chunk is upserted as it arrives, so a mid-window failure leaves the earlier chunks persisted and the next run resumes further along instead of repeating the whole doomed request.

Semantics preserved exactly (they are load-bearing and commented):

• The ±step widening applies to the OUTER edges only — venues are inconsistent about endpoint inclusivity. Internal chunk boundaries stay contiguous and are not themselves widened.
• A window is eligible to be recorded absent at source only if EVERY chunk completed. That is the difference between "the venue lacks this data" and "we failed to ask properly", and the module docstring is explicit that the absence record is an ASSERTION ABOUT AN OBSERVATION. A partial failure must never let a window be recorded absent.
• The per-window except ... continue stays, so one bad window cannot abort the pass.
• sleep_fn paces every chunk request, matching history._fill_forward. cli.py passes the real time.sleep, so this is real rate limiting.

repair_series's signature is unchanged (keel/cli.py and tests/data/test_fetch_cli.py depend on it).

Defect 2 — history.py requested 301 candles, not 300

window_end   = min(now_ts,      window_start + MAX_CANDLES_PER_REQUEST * step)
window_start = max(start_floor, window_end   - MAX_CANDLES_PER_REQUEST * step)

An INCLUSIVE range [a, a + N*step] holds N+1 candles. Verified: _fill_forward over a 1000-candle gap produced window sizes [301, 301, 301, 97].

Harmless today (301 < 350) — but history.py says MAX_CANDLES_PER_REQUEST = 300 # conservative under Coinbase's ~350-candle response cap, which openly invites raising it toward 350, at which point this silently requests 351 and reproduces the incident in the backfill path.

Both sites now use (MAX_CANDLES_PER_REQUEST - 1) * step — giving [300, 300, 300, 100] — matching what #269 did in market_feed.py, with a comment on the arithmetic at each site so the -1 never reads as an accident.

Tests (written first, confirmed failing for the right reason)

Reverting only the keel/ changes while keeping the new tests:

FAILED tests/data/test_gap_repair.py::test_a_large_hole_is_fetched_in_capped_chunks
FAILED tests/data/test_gap_repair.py::test_a_large_hole_is_fully_recovered_with_no_duplicates_or_drops
FAILED tests/data/test_gap_repair.py::test_chunk_windows_tile_the_outer_range_without_widening_interior_boundaries
FAILED tests/data/test_gap_repair.py::test_one_bad_chunk_is_not_recorded_absent_and_a_later_window_still_repairs
FAILED tests/data/test_gap_repair.py::test_sleep_fn_is_called_once_per_chunk_not_once_per_window
FAILED tests/data/test_gap_repair.py::test_a_multi_chunk_window_with_nothing_at_venue_is_still_recorded_absent
FAILED tests/data/test_history.py::test_fill_forward_never_requests_more_than_the_cap_per_call
FAILED tests/data/test_history.py::test_fill_backward_never_requests_more_than_the_cap_per_call
FAILED tests/data/test_history.py::test_window_sizing_tracks_the_cap_constant_so_raising_it_toward_350_stays_safe
9 failed, 24 passed in 0.78s

Every failure is a real assertion about candle counts, call counts or absence records — assert 351 <= 350 where 351 = max([351, 351, 298]), assert sizes == [300,300,300,100] vs actual [301,301,301,97], assert len(venue.calls) == 2 vs actual 1 — not an import, fixture or typo error. The tenth new test (test_a_small_hole_still_takes_exactly_one_request) is a regression guard that passes both before and after, by design.

The repair tests drive an oversized (553-candle) gap against a _CappedVenue fake that enforces the true ~349 ceiling, so the fake actually returns the 400 the real venue would. They assert on the actual start/end args it received (chunks of exactly [300, 255]), that the chunks tile the outer range with no duplicate and no gap at a boundary, that all 553 rows land, that a failed chunk is not recorded absent while a later window in the same pass still repairs, and that a fully-completed multi-chunk window with nothing at the venue is still recorded absent.

The history tests assert exact window sizes and express the invariant in terms of the constant — one monkeypatches MAX_CANDLES_PER_REQUEST to 350 and asserts no request exceeds it — which is what makes raising the constant safe.

Gates

uv run ruff check keel tests packages   All checks passed!
uv run mypy                             Success: no issues found in 224 source files
uv run pytest -q                        2737 passed, 1 skipped in 31.11s

Baseline at 29d9b20 measured directly: 2727 passed, 1 skipped, plus the 10 added here.

Note this branch is cut from main, independent of #269 — they touch disjoint files (market_feed.py vs repair.py/history.py) and do not conflict.

🤖 Generated with Claude Code