Skip to content

fix(fetch): window-scope the gap-proven display — the two counts must share a window - #330

Merged
eaitbrahim merged 2 commits into
mainfrom
fix/gap-display-window
Aug 17, 2026
Merged

fix(fetch): window-scope the gap-proven display — the two counts must share a window#330
eaitbrahim merged 2 commits into
mainfrom
fix/gap-display-window

Conversation

@eaitbrahim

Copy link
Copy Markdown
Contributor

What & why

keel fetch printed, for real series today (2026-08-17):

GAPS  SOL-USD  ONE_HOUR  n=43640  1 bars behind, 158 internal gaps (-2 proven absent at venue)
GAPS  SOL-USD  ONE_DAY   n=1819   0 bars behind, 5 internal gaps (-1 proven absent at venue)

A negative "proven absent" count is an impossible claim — you cannot prove fewer than zero
gaps absent — and it MASKED the real state: for those series the whole-series unexplained-gap
count was 160 hourly / 6 daily (some gaps are NOT proven absent), plus 2+1 missing bars sit
OUTSIDE the assessed window, yet the display read as if reconciliation had happened.

Root cause: _assess_products computed coverage()'s gap count over a WINDOW-BOUNDED read
(get_candles(product, gran, start_ts, None)) but unexplained_gap_count() over the WHOLE
series, then _print_freshness subtracted one from the other. Any bar missing older than
start_ts drove the suffix negative.

The fix makes the two counts share a window by construction:

  • unexplained_gap_count gains an optional start_ts (default None = whole series,
    behavior identical to today). Given start_ts, it reads the same bounded slice
    coverage() reads, so detection covers the same bars. The docstring records the honest
    boundary caveat: a hole that straddles start_ts is not interior to the bounded slice, so
    neither count can see it — the window can only under-report a hole crossing its own start
    boundary, never claim one absent, and the whole-series default still sees it.
  • _assess_products passes start_ts; _print_freshness keeps proven = gaps - unexplained
    with NO max(0, ...) clamp — the point is the shared window, and a clamp would only re-hide
    a future mismatch.

--fail-on-gaps scope deliberately UNCHANGED (whole series). Reading the fetch flow:
the flag judged _assess_products' whole-series return, and the advice it prints (run keel fetch --repair-gaps) points at repair_series, which itself reads the whole series — so a
hole older than the fetch window is still fixable, still unproven, and still the flag's
business. The flag now computes its whole-series count where it is judged (the --check
branch) instead of riding the display's window-bounded number, and a test pins that a gap
lying entirely OUTSIDE the fetch window still fails --check --fail-on-gaps (and that plain
--check still reports it in the closing "N have UNEXPLAINED gaps" message). The default of
unexplained_gap_count stays whole-series and is pinned by test.

Tests-first evidence

Red first (uv run pytest tests/data/test_fetch_cli.py tests/data/test_gap_repair.py -q):

4 failed, 40 passed in 1.36s

Failing for the right reasons:

  • test_the_gap_suffix_shares_the_fetch_window_so_it_cannot_go_negative — assertion failure
    whose diff SHOWS the bug (the exact old rendered line):
    GAPS SOL-USD ONE_DAY n=363 0 bars behind, 2 internal gaps (-1 proven absent at venue)
  • test_every_assessed_row_keeps_proven_absent_never_negativeAssertionError: ('SOL-USD', ONE_DAY, 2, 3): the field shape (window gaps 2 < whole-series unexplained 3).
  • test_unexplained_gap_count_bounded_ignores_holes_before_start_ts and
    test_a_hole_straddling_start_ts_is_invisible_to_the_bounded_count
    TypeError: unexplained_gap_count() got an unexpected keyword argument 'start_ts' (the
    interface under test not existing yet).
  • test_fail_on_gaps_still_judges_holes_older_than_the_fetch_window was BORN GREEN, on
    purpose and stated in its docstring: it pins preserved --fail-on-gaps semantics, not the
    regression — it must pass both before and after or the flag's scope changed.

Fixture = the field repro: (a) an in-window hole recorded absent in candle_gap_probes,
(b) an in-window hole NOT recorded, (c) a 2-bar hole entirely older than start_ts.
Before → after, exact rendered line:

- GAPS     SOL-USD      ONE_DAY   n=363     0 bars behind, 2 internal gaps (-1 proven absent at venue)
+ GAPS     SOL-USD      ONE_DAY   n=363     0 bars behind, 2 internal gaps (1 proven absent at venue)

The suffix now claims exactly the one proven bar; the unproven in-window hole rides in the
2 internal gaps count unclaimed; the outside-window hole perturbs neither number, and the
closing message still carries the whole-series truth (1 have UNEXPLAINED gaps).

  • Tests written first, seen failing for the right reason

Gates (all must pass)

  • uv run ruff check clean — All checks passed! (over keel tests packages)
  • uv run mypy clean — Success: no issues found in 237 source files
  • uv run pytest -q green — 2862 passed, 1 skipped in 36.56s

Scope check

  • This PR touches a rail or a default classification — unchecked: display + one pure
    function's optional parameter only. No rail, no rule, no classification, no broker
    surface, no default changed (--fail-on-gaps whole-series semantics preserved and
    pinned by test).
  • New dependency added — none.

@eaitbrahim
eaitbrahim merged commit 93db046 into main Aug 17, 2026
5 checks passed
@eaitbrahim
eaitbrahim deleted the fix/gap-display-window branch August 17, 2026 19:09
@eaitbrahim eaitbrahim mentioned this pull request Aug 17, 2026
4 tasks
eaitbrahim added a commit that referenced this pull request Aug 17, 2026
Version bump across all six distributions. Patch: exactly one change
since v0.8.0 (#330), the window-scoped gap-proven display in keel fetch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant