Repository navigation
fix(stack): bound terminal GitHub delivery history - #4826
ScriptedAlchemy wants to merge 26 commits into
Conversation
|
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Blocking retention-work finding: LIMIT 16 does not bound the writer scan
At 83614c4, retire_github_stack_terminal_payloads_before opens transact_or before querying all old unretired signals and applying NOT EXISTS against pending/deferred recipients. LIMIT 16 bounds returned payloads, not examined signals. The observed_at_micros partial index still includes every old pending/deferred signal, so a long saturated backlog is rescanned under the SQLite writer on every slice and even when nothing can be reclaimed. The same prefix can be rescanned up to the tick's 64 slices. This is especially risky given the current retention CPU/writer-contention reports.
Independent executable reproduction used the literal candidate SQL and the current STRICT tables/three indexes on SQLite 3.46.1. One recipient has the allowed 256 pending rows and the remainder deferred; all are older than the cutoff. With exactly 16 later eligible settled signals:
| Old ineligible signals | Returned signals | SQLite VM steps inside writer |
|---|---|---|
| 256 | 16 | 5,204 |
| 4,096 | 16 | 78,164 |
| 65,536 | 16 | 1,245,524 |
With no eligible signals, the same sizes cost 4,626 / 77,586 / 1,244,946 VM steps while returning zero. These are SQL-mechanism measurements, not Rust/Bazel or production CPU evidence.
Before merge, make candidate discovery bounded by inspected rows as well as returned signals, retain forward progress across pending prefixes/restarts, and revalidate eligibility atomically before mutation. In particular, don't replace this with LIMIT before eligibility without a progress frontier: that would permanently starve terminal rows after an old pending prefix. Keep newly settled earlier signals eventually discoverable and preserve the current transaction/rollback/replay protections. Add a real registered-store regression with a large old deferred prefix, eligible rows beyond it, idle passes, and writer-VM/admission measurements; verify no full-backlog writer scan. Do not raise deadlines or shrink the fixture to hide this.
Keep this PR draft and #4445 open. This review does not approve or merge it.
Refs #4445. Fail closed on malformed retired markers and keep exact replay recognition independent of whitespace. Reject nonhex retirement fingerprints at the SQLite immutable-signal trigger. Adds registered-store marker/replay/SQL refusal regression.
Refs #4445. SQLite accepts Unicode-escaped marker keys; decode JSON before checking the retirement identity so an escaped key cannot reopen a compacted signal. Add registered-store regression.
Refs #4445. Reject oversized multibyte JSON at the registered SQL boundary, with a real store regression for refused raw writes and accepted bounded content.
Refs #4445. Install the UTF-8 ceiling trigger without rebuilding existing tables, preserving historical constraint and ensuring raw inserts cannot exceed the canonical payload bound.
Refs #4445. Raw SQL INSERT must not bypass the reserved retirement marker policy enforced at application admission. Reject forged, null, Unicode-escaped and duplicate marker keys without changing real terminal retirement updates.
Stack deliveries are written to each project session shard, but the automation scheduler retired only the profile database and only while an automation job kept the project scheduler alive. Run the bounded, frontier-backed retirement from the registered-store retention kernel, which the daemon maintenance tick already admits for the profile and every mounted session shard, and remove the scheduler copy. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Stack delivery history has a fixed inspection window that no operator policy controls, but the maintenance loop was skipped entirely when every optional session, observation, orphan-store and compaction policy was disabled. Admit the existing loop in a fixed mode in that case: it retires terminal deliveries from each mounted session shard and runs none of the full tick's optional work. Soft budgets still never count as a policy. Cover the daemon journey on a project with every agent job disabled, including the wrong-shard profile cleanup and the 64-page budget resumed from its durable frontier. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Expired settled GitHub stack signals retained full transport payloads indefinitely. This change retires them after the fixed 14-day inspection window while preserving exact replay identity and pending delivery history. Refs #4445.
This PR depends on #4834 at
5618857159c87674533f8d11b407741d70de4763; its source branch was merged normally, and this PR is based oncodex/cli-transport-fixture-admission. Integrate #4834 before this PR.Validation on macOS through Bazel:
Acceptance limits: fixed housekeeping covers mounted shards only. Hosted checks and the prerequisite's existing CPU/LSP qualification holds remain separate acceptance work. This validation does not claim native-host expansion, durable host receipt readback, or production-volume CPU qualification. Keep #4445 open; no PR merge is authorized here.