Report a task-wide gap from a batching connector too (#193)

[richardmhope]

Closes #193.

The defect

When a connector declares CHECKPOINTS, the engine flushes findings and a resume position mid-fetch, and its terminal submission carries only the remainder the progress batches did not (#143). The task-wide scope gap recorded when a fetch dies partway — a rate limit, a timeout, an unanticipated connector error — was written onto the whole-task tally, which that submission no longer sent. The gap never reached the API.

The order is what made it invisible. _fetch's finally froze the remainder while the exception was still unwinding; _record_task_gap ran afterwards, in run_task's handler, and wrote into a field as_submission had stopped reading.

Downstream, build_manifest uses a fetch task's scope.gaps to decide it cannot claim a known scope size (fetch_scope_gap, apps/api/src/iceberg_api/scans/coverage.py). With no gap it goes on reporting one — so a scan that read an unknown fraction of a Confluence space, a Jira project, or a file-share root presented as having read all of it. That is the specific thing coverage manifests exist to prevent.

The same failure against a connector without checkpoints reported the gap correctly. Two connector classes told different stories about the same event, which is how it surfaced.

Reproduction

A connector that yields three pages and then raises RateLimitError, run twice — once resumable, once not:

	scope.gaps	reasons
not resumable	1	[{"outcome": "scope_gap", "reason": "rate_limited", "count": 1}]
resumable (batches)	0	[]

The fix

TaskReport now holds the batches' baseline (sent_coverage) rather than a pre-computed delta, and as_submission derives the remainder — after _record_task_gap has run. coverage stays the whole task's tally for its whole life, which is what the field already meant everywhere else.

This is smaller than the alternative of folding a gap into an existing delta, and it removes the special case rather than adding one: both connector classes now take the same path through _record_task_gap, and the gap-reference cap (MAX_COVERAGE_GAP_REFERENCES) applies to the task's real list rather than to a freshly-created one that always had room.

Valid because a delta is only ever cut at a unit boundary and gaps is append-only, so the gap appended last lands in the tail _coverage_delta already slices.

Tests

Three in apps/engine/tests/test_runner_batching.py, each confirmed to fail against the pre-fix code:

• the gap reaches the API after the batches that went before it — and the submission is still a delta, so the fix does not re-report what the batches carried;
• the two connector classes report the same gap, reason, and HMAC reference for the same failure;
• the time limit's BaseException path, which submits from its own handler, carries it too.

make check is green: ruff, mypy (278 files), docs check, 1953 passed / 2 skipped (the two file-share permission tests that cannot run as root).

Operator impact

Manifests already written are not recomputed; the CHANGELOG entry says so and points at re-running the scan.

---

Generated by Claude Code

[icebergai-review-bot]

Verdict

APPROVE

Completed bounded review across 1 immutable scope(s). No actionable regressions found in the supplied change.

Scope health

Convergence: healthy. Review mode: initial.
Recommended action: CONTINUE_INCREMENTAL.

• No escalation signals.

Prior findings

Finding	Status
—	No prior finding state

New findings

No new findings.

Fix-induced regressions

• None evidenced.

Uncertainty

• No material uncertainty recorded.

Validation

• Reviewed the supplied immutable diff, including batching and failure-path coverage reporting changes and their tests.

Residual risks

• None identified.