Skip to content

fix(stack): bound terminal GitHub delivery history - #4826

Open
ScriptedAlchemy wants to merge 26 commits into
codex/cli-transport-fixture-admissionfrom
fix/4445-stack-terminal-retention
Open

ScriptedAlchemy wants to merge 26 commits into
codex/cli-transport-fixture-admissionfrom
fix/4445-stack-terminal-retention

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

Expired settled GitHub stack signals retained full transport payloads indefinitely. This change retires them after the fixed 14-day inspection window while preserving exact replay identity and pending delivery history. Refs #4445.

This PR depends on #4834 at 5618857159c87674533f8d11b407741d70de4763; its source branch was merged normally, and this PR is based on codex/cli-transport-fixture-admission. Integrate #4834 before this PR.

  • Each registered transaction examines at most 64 ordered identities and retires at most 16 eligible payloads. It commits its durable frontier atomically with payload fingerprinting and terminal-recipient deletion. Pending/deferred recipients, recent signals, replay guards, and rollback protections remain intact.
  • The daemon admits one terminal page per store unit through the prerequisite's supervised canonical store jobs. Writer and CPU admission release after that unit; unrelated stores can write while another store waits in SQLite. No retained unpolled acquisition futures or separate housekeeping scheduler remain.
  • Successful unfinished pages return the existing StoreRetention continuation. Continuation advances terminal and lossless storage work without replaying optional age policies; typed storage errors still return Retry.
  • With all automation and optional retention policies disabled, the existing maintenance owner runs only fixed terminal housekeeping over mounted session shards. Soft budgets do not enable optional work. The shared eight-store round-robin window retains continuation for unvisited shards and holds no project leases between units.

Validation on macOS through Bazel:

  • Two native regressions failed before the repair on the intended behavior: unrelated B writer admission blocked behind A, and optional age deletion replayed after successful bounded terminal progress. Both pass after the repair.
  • 19 daemon maintenance tests, five terminal-housekeeping tests, five production capacity journeys, nine writer-gate tests, and 160 maintenance-kernel tests passed. The unchanged global-database suite's 508-test pass was reused from Bazel's cache.
  • Terminal journeys cover wrong-profile isolation, disabled automation/policies, the 1,041 settled + three pending backlog across page admissions, durable frontier drain, nine mounted shards across bounded windows, and typed failure with transaction rollback. Shared admission tests cover fairness, cancellation, identity replacement, between-tick completion, and cooperative shutdown.
  • Affected production libraries, workspace rustfmt, strict Clippy for both affected unit-test targets, generated Bazel targets, and dashboard/SDK contract checks passed.

Acceptance limits: fixed housekeeping covers mounted shards only. Hosted checks and the prerequisite's existing CPU/LSP qualification holds remain separate acceptance work. This validation does not claim native-host expansion, durable host receipt readback, or production-volume CPU qualification. Keep #4445 open; no PR merge is authorized here.

@changeset-bot

changeset-bot Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 4baadf2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking retention-work finding: LIMIT 16 does not bound the writer scan

At 83614c4, retire_github_stack_terminal_payloads_before opens transact_or before querying all old unretired signals and applying NOT EXISTS against pending/deferred recipients. LIMIT 16 bounds returned payloads, not examined signals. The observed_at_micros partial index still includes every old pending/deferred signal, so a long saturated backlog is rescanned under the SQLite writer on every slice and even when nothing can be reclaimed. The same prefix can be rescanned up to the tick's 64 slices. This is especially risky given the current retention CPU/writer-contention reports.

Independent executable reproduction used the literal candidate SQL and the current STRICT tables/three indexes on SQLite 3.46.1. One recipient has the allowed 256 pending rows and the remainder deferred; all are older than the cutoff. With exactly 16 later eligible settled signals:

Old ineligible signals Returned signals SQLite VM steps inside writer
256 16 5,204
4,096 16 78,164
65,536 16 1,245,524

With no eligible signals, the same sizes cost 4,626 / 77,586 / 1,244,946 VM steps while returning zero. These are SQL-mechanism measurements, not Rust/Bazel or production CPU evidence.

Before merge, make candidate discovery bounded by inspected rows as well as returned signals, retain forward progress across pending prefixes/restarts, and revalidate eligibility atomically before mutation. In particular, don't replace this with LIMIT before eligibility without a progress frontier: that would permanently starve terminal rows after an old pending prefix. Keep newly settled earlier signals eventually discoverable and preserve the current transaction/rollback/replay protections. Add a real registered-store regression with a large old deferred prefix, eligible rows beyond it, idle passes, and writer-VM/admission measurements; verify no full-backlog writer scan. Do not raise deadlines or shrink the fixture to hide this.

Keep this PR draft and #4445 open. This review does not approve or merge it.

Refs #4445. Fail closed on malformed retired markers and keep exact
replay recognition independent of whitespace. Reject nonhex retirement
fingerprints at the SQLite immutable-signal trigger.

Adds registered-store marker/replay/SQL refusal regression.
Refs #4445. SQLite accepts Unicode-escaped marker keys; decode
JSON before checking the retirement identity so an escaped key
cannot reopen a compacted signal. Add registered-store regression.
Refs #4445. Reject oversized multibyte JSON at the registered SQL boundary, with a real store regression for refused raw writes and accepted bounded content.
Refs #4445. Install the UTF-8 ceiling trigger without rebuilding existing tables, preserving historical constraint and ensuring raw inserts cannot exceed the canonical payload bound.
Refs #4445. Raw SQL INSERT must not bypass the reserved retirement marker
policy enforced at application admission. Reject forged, null, Unicode-escaped
and duplicate marker keys without changing real terminal retirement updates.
@ScriptedAlchemy
ScriptedAlchemy marked this pull request as ready for review November 9, 2026 23:55
devin-ai-integration[bot]

This comment was marked as resolved.

ScriptedAlchemy and others added 2 commits November 9, 2026 22:25
Stack deliveries are written to each project session shard, but the
automation scheduler retired only the profile database and only while an
automation job kept the project scheduler alive. Run the bounded,
frontier-backed retirement from the registered-store retention kernel,
which the daemon maintenance tick already admits for the profile and every
mounted session shard, and remove the scheduler copy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Stack delivery history has a fixed inspection window that no operator
policy controls, but the maintenance loop was skipped entirely when every
optional session, observation, orphan-store and compaction policy was
disabled. Admit the existing loop in a fixed mode in that case: it retires
terminal deliveries from each mounted session shard and runs none of the
full tick's optional work. Soft budgets still never count as a policy.

Cover the daemon journey on a project with every agent job disabled,
including the wrong-shard profile cleanup and the 64-page budget resumed
from its durable frontier.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
devin-ai-integration[bot]

This comment was marked as resolved.

@ScriptedAlchemy
ScriptedAlchemy changed the base branch from master to codex/cli-transport-fixture-admission November 10, 2026 07:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant