Repository navigation
fix(lcm): preserve retry backoff across dirty invalidation - #4808
ScriptedAlchemy wants to merge 13 commits into
Conversation
Keep retryable and terminal outcomes durable when an earlier raw revision still has invalidation work; only new revisions can rearm those failures. Cover retry deadlines across restart, terminal parking, recovery on new raw input, and stale-generation refusal. Refs #4481
|
Repair three hosted automation_runner_test failures by supplying the configured executable identity required by the production prefilter. Use the same identity for the prefilter and retained runner, preserving manual-schedule, exact-ledger, live-lock and lock-race assertions. Add a production-prefilter regression for unconfigured refusal, deduplication and same-occurrence recovery after binding, with zero backend calls and no effectful terminal. Do not relax backend admission, change timeouts, or alter the graph publication changes. Refs #4532
There was a problem hiding this comment.
Note
Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
|
Independent exact-head review blocker for #4481 (head |
Restore bounded incremental reclamation only after short-lived schema convergence. The daemon admission path stays noncompacting; refused stores retain their byte-preserving refusal. Cover physical page-count reduction after an actual writable reopen. Refs #4481
|
Follow-up to the short-lived-store reclamation review finding: published |
A short-lived registered store has no daemon compactor, but a failed incremental vacuum after schema convergence is optional maintenance, not a reason to reject a valid database owner. Report the failure and allow a later open to retry page reclamation without masking required schema-convergence errors. Refs #4481
Only SQLite vacuum contention may defer short-lived admission. Thread typed compaction deferral through the runtime and retain fatal writer, reply, and authority failures; cover classification in regression. Refs #4481
|
Published Independent root-cause review confirmed the writer-admission finding: the prior The new patch distinguishes deferrable SQLite-vacuum failure / busy maintenance queue from fatal writer shutdown, failed replies, and revoked write authority at the writer, repository, and store-runtime boundaries. Short-lived admission defers only typed maintenance contention; writer/authority failures now propagate. The regular required-compaction entry point still propagates every failure. A focused error-classification regression checks fatal writer and authority cases versus deferrable vacuum cases. The existing real-registered-store regression continues to check physical page/freelist reduction after a short-lived reopen. Verification limit: the local execution environment has no Bazel or native macOS daemon. These Rust tests have not been executed locally; exact-head CI 42544645963 is pending. A true end-to-end writer-revocation admission regression remains additional desirable coverage; the classification test is not a substitute for the daemon CPU or 24-hour bounded-growth acceptance. Master has independently advanced to |
|
Refs #4481 — new exact-head CI blocker, not a diagnosed #4481 source regression (PR head CI run 42549652638, Linux fleet job 128953370614 finished 182/183 Bazel targets passing, including
The prior master fleet run at Keep merge held. There is still no verified native macOS idle-CPU or 24-hour bounded-store daemon acceptance on master. No merge/closure or CI-rule relaxation is warranted. |
|
Refs #4481 — new exact-head CI evidence on The parent run still has Linux and Windows work queued/in progress, so the appropriate next action is targeted retry of the failed job after the run completes, keeping the pinned dependency, CI policy, and tests unchanged. No source change, CI bypass, merge, or #4481 closure is justified by this attempt; native macOS daemon idle CPU, retry convergence, and 24-hour store bounds remain to be proven on master. |
…aults Refs #4481 Classify SQLite FULL on optional incremental vacuum as a maintenance failure, not lost writer authority. Preserve fatal read-only, closed writer and revoked-authority refusal. Exercise an actual SQLite FULL quota error and the repository dispatch boundary.
|
Refs #4481 — published Additional root cause: the earlier classification fix only deferred SQLite The focused three-file patch classifies only Independently reviewed the resulting three-file commit diff and parent ( |
Refs #4481 Rebase the accepted 13-file issue-specific patch intact onto current master without rewriting any existing owner branch. Persist retry and terminal settlement across dirty LCM invalidation, bound store page reclamation, and preserve fatal writer/authority failures while deferring optional SQLite BUSY, LOCKED, or FULL compaction. This clean successor replaces the blocked PR #4808 branch, whose last commit header exceeded the enforced 72-character limit. Carry the exact same 13 file blobs and preserve unrelated master changes.
|
Refs #4481 — superseded by #4851, not merged. Exact-head Repository gates rejected this head because its last commit header is 74 characters versus the enforced 72-character maximum; Feature gates then correctly failed. Amending this head would violate the non-force/exact-head branch-preservation requirement. #4851 preserves all 13 exact file blobs and the exact same per-file diff stats from this PR, on current master in one valid 57-character-header commit; there was no file overlap with master's intervening eight commits. Successor CI is queued, not passed. The previous branch remains intact for review/history, but this duplicate PR is closed to avoid wasting CI and accidentally landing the blocked commit history. Do not close #4481: native macOS idle CPU, repeated retry convergence and 24-hour bounded-store acceptance remain unproven on master. |
Pull request was closed
Scope
Refs #4481. A retained-summary invalidation may fail while the raw revision remains dirty. Convergence now persists the retry deadline or terminal state for that revision. A genuinely new revision increments its generation, wakes the session and prevents a stale worker from settling newer work. A successful page still leaves outstanding dirty input pending.
Daemon attachment leaves ordinary free-page reclamation to maintenance. Short-lived registered-store owners lack that maintenance turn and reclaim up to 4,096 pages after successful admission. A full optional maintenance queue or a SQLite busy or locked result defers reclamation. A rejected writer, lost authority, read-only database, or other SQLite failure prevents a serving lease. The workflow artifact migration retains its explicit post-upgrade reclamation. The branch also includes the scheduler backend-identity and scripted CLI daemon-binding test fixtures.
Verification
On the source now published at
18c4422c2d, native macOS Bazel passed all 428 rusqlite-runtime tests, all 499 global database tests, and 187 LCM tests (one ignored). Four existing summary-convergence cases and the added end-to-end failure case passed. The six vacuum cases also passed in isolation. The previous head’s hosted formatting failure inrepository/attachment.rshas been corrected.A production convergence-page regression injects a SQLite failure at the final dirty-revision deletion. It checks that the transaction leaves the same revision intact, that the queue delays the next attempt, and that a fresh revision wakes the session and clears the dirty row after the database recovers. The SQLite writer tests exercise real lock contention, a read-only connection, worker rejection, and revoked authority. Writer admission review
The hosted Repository gates and Clippy checks passed. The Linux fleet job reported two five-second settlement deadlines after the durable Prepared phase. The same two cases failed at the same assertions on a different branch, and both passed in the #4791 Linux suite. The automations themselves use isolated stores, and the exact publication code is identical on those heads.
On the unchanged #4808 head, all nine retained-settlement cases passed three times on native macOS. With temporary timing probes, all 437 automation tests passed three more times. The two affected tests each published their durable ledger entry in 69–95 ms across those runs. The artifact provides no per-phase Linux timings. A local experiment removing one directory sync did not reduce the observed upper bound. The experiment was discarded. The subsequent exact-head hosted rerun passed, as recorded below.
Remaining acceptance
The affected macOS host still needs idle CPU, retry convergence, and 24-hour growth measurements on master. #4481 remains open pending that evidence and independent review.
The hosted Linux fleet rerun passed all 183 Bazel test targets on the unchanged head
18c4422c2d. The automation runtime unit target executed in 71.5 seconds, including both previously timed-out settlement cases. The preceding rerun stopped before tests when GitHub returned HTTP 504 for the pinned launcher download. The original #4481 master acceptance remains open.Mixed fixture ownership review
At
4cee32eb43, the PR incorporates the accepted master snapshot02e3c8b28b. The shared scheduler fixture and scripted CLI transport fixture both match master. The canonical CLI fixture remains owned by #4834. The proposed diff now consists of 13 LCM/storage production and test paths, all byte-identical to their versions on the previously green18c4422c2dhead.The session-runtime LCM regression belongs here. It fails the final dirty-row deletion during production convergence, observes that the same input stays delayed, and verifies that a changed revision is admitted and clears the durable dirty row after recovery. The prior mixed-fixture review is addressed. Current-head CI is pending.
On the isolated head
4cee32eb43, all nine focused native macOS Bazel cases passed: the production dirty-revision transaction regression, the LCM restart/backoff regression, short-lived registered-store physical page reclamation, and six vacuum contention, refusal, or page-budget cases. Hosted repository gates also passed. Keep #4481 open for its separate master acceptance.