fix(daemon): recover typed CAS frontier failures - #3918
Conversation
Problem: stopped-daemon raw recovery ignored canonical and historical CAS frontier evidence even though the production repair selector accepted both as replay authority. What changed: correlate the daemon probe with the exact raw-artifact coordinate and existing typed replay-authority vocabulary. Add a temporary-SQLite route test that persists the failure through ArchiveStore and drains it through the daemon recovery stage and repair engine. Ref polylogue-dyica and polylogue-dyica.1. Co-authored-by: Codex <noreply@openai.com>
📝 WalkthroughWalkthroughRaw parse recovery now treats deferred raw-authority failure evidence as pending work. Parametrized tests cover canonical and historical CAS-frontier failures and verify that recovery materializes the affected raw row. ChangesRaw parse recovery
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8e194d2613
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| AND failure_evidence.artifact_kind IN ({replay_authority_placeholders}) | ||
| AND failure_evidence.support_status = ? |
There was a problem hiding this comment.
When an exact CAS carrier belongs to a raw whose validation_status is failed, this new EXISTS marks the path pending even though the executor explicitly rejects such rows in storage/repair.py:3952-3955. Consequently, execute performs no repair, recomputes the same positive count, and returns False, so this stage's false_means_pending behavior preserves and repeatedly retries convergence debt indefinitely. Mirror the executor's validation-status predicate in this probe so only raws actually authorized for replay are selected.
AGENTS.md reference: AGENTS.md:L168-L177
Useful? React with 👍 / 👎.
| OR r.parse_error = 'OperationalError: database is locked' | ||
| OR r.parse_error LIKE 'decode:%No such file or directory:%' | ||
| OR r.parse_error LIKE 'membership_replay_conflict:%' | ||
| OR EXISTS ( |
There was a problem hiding this comment.
When replaying a previously successful raw after its derived index row is lost, a RawCASFrontierError leaves the old parsed_at_ms intact because _raw_parse_failure_state does not update that field. Although repair.py deliberately admits these already-parsed, unmaterialized rows, this new typed-evidence branch remains inside the outer r.parsed_at_ms IS NULL condition, so the stopped-daemon probe reports no pending work and never invokes the repair engine. Move the CAS-authority case outside that parsed-at restriction, while retaining the materialization checks.
Useful? React with 👍 / 👎.
Problem: The stopped-daemon pending probe could retain validation-failed CAS rows and skip previously parsed, unmaterialized CAS rows, unlike the repair selector it triggers. What changed: Mirror failed-validation eligibility and separate unparsed legacy retries from typed CAS retry authority. File-backed regressions cover both routes. Ref polylogue-dyica. Co-Authored-By: Codex <noreply@openai.com>
|
@codex review |
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
## Summary Record the verified closure of source-index ground-truth and testmon seed-contract work. ## Problem The implementation PRs are merged, but the authoritative Beads export still reported their residuals as open. ## Solution Close the four satisfied verification residuals with exact merged-PR and verification evidence. Parent raw-failure/live remediation remains open. The separate audit-toolchain Bead remains open because its full dependency roster is not yet present. ## Verification - PR #3919 merged at `a9744ae1b`; focused red-twin and quick gate passed. - PR #3918 merged at `ccb73aa3e`; focused raw-recovery tests and quick gate passed. - PR #3920 merged at `8a388e420`; 162 focused harness tests and quick gate passed. - Fresh seed receipt is typed `resource-timeout` and explicitly not release-authorizing. - `polylogue-29hwx` was deliberately reopened after review found its audit dependency roster incomplete; it is not assigned to this closure PR. <!-- polylogue-pr-scope:v1 { "assigned_beads": [ "polylogue-r4jiu", "polylogue-dyica.1", "polylogue-817er", "polylogue-d96ta" ], "beads_digest": "2baf0ca8a36d7ddce540f7d34a708f01692a48bb8af1791fca8f5d48e55ff424", "dispositions": [ {"bead_id":"polylogue-r4jiu","disposition":"satisfied","evidence":[{"kind":"commit","ref":"a9744ae1b merged PR #3919"},{"kind":"test","ref":"source-index coverage red twin and focused registry test passed"},{"kind":"command","ref":"devtools verify --quick: exit 0"}],"successors":[]}, {"bead_id":"polylogue-dyica.1","disposition":"satisfied","evidence":[{"kind":"commit","ref":"ccb73aa3e merged PR #3918 typed CAS recovery implementation"},{"kind":"test","ref":"raw parse recovery focused suite and quick gate passed"}],"successors":[]}, {"bead_id":"polylogue-817er","disposition":"satisfied","evidence":[{"kind":"commit","ref":"8a388e420 merged PR #3920"},{"kind":"test","ref":"162 focused testmon harness tests passed"},{"kind":"command","ref":"devtools verify --quick: exit 0"}],"successors":[]}, {"bead_id":"polylogue-d96ta","disposition":"satisfied","evidence":[{"kind":"receipt","ref":"typed fresh-seed resource-timeout receipt bound to exact tree/dependencies/harness/node universe"},{"kind":"command","ref":"POLYLOGUE_VERIFY_PYTEST_TIMEOUT_S=600 devtools verify --seed-testmon: exit 124; release_baseline_allowed=false"}],"successors":[]} ], "beads_digest": "2baf0ca8a36d7ddce540f7d34a708f01692a48bb8af1791fca8f5d48e55ff424", "head_sha": "8867a00896777c2bf183d6e5f0e9dc11dcc7f561", "scope_digest": "655aa4b15af02a0b57c9f0427062fd946c2ac2183ee2035d34bb9a76c60189a5", "version": 1 } -->
1 participant
Summary
Route typed CAS-frontier failures into stopped-daemon raw recovery only when the existing repair selector authorizes replay. The probe recognizes canonical and historical CAS carriers by exact raw-artifact coordinate, admits previously parsed but unmaterialized raws, and delegates materialization to the existing repair engine.
Problem
raw_materializationalready selecteddeferred_cas_frontieranddeferred_codex_cas_frontieras retry authority. The daemon's path-scopedraw_parse_recoveryprobe initially recognized only untyped error strings. Its first typed-carrier version then diverged from the executor in two ways: validation-failed raws stayed pending forever, while previously parsed but unmaterialized CAS raws were skipped.Solution
polylogue/daemon/convergence_stages.pyimports the established core replay-authority vocabulary and requires the same raw ID, origin, source path, source index, partial-decode support status, failed-validation exclusion, and materialization guard before returning pending work. It keeps legacy raw retries unparsed-only while allowing typed CAS authority for previously parsed raws.tests/unit/daemon/test_raw_parse_recovery.pycreates real file-backed temporary archive tiers, writes authority and parse transitions throughArchiveStore, and exercisesraw_parse_recoverythrough the production repair engine until the session materializes.polylogue-dyica08c363b0d; stopped-daemon route regressions; quick gatepolylogue-reindex-source-remediation.polylogue-dyica.1polylogue-dyicaand remains open aspolylogue-reindex-source-remediation; no live apply is claimed here.Verification
devtools test tests/unit/daemon/test_raw_parse_recovery.py -k 'validation_failed_cas_frontier_failure or previously_parsed_cas_frontier_failure'failed before this correction with two inversestage.check(path)assertions.devtools test tests/unit/daemon/test_raw_parse_recovery.pythen reported19 passed.devtools verify --quickcompleted all 24 checks with exit 0. The suite uses real temporarysource.dbandindex.dbfiles, not an in-memory substitute. Reverting either eligibility condition restores its corresponding production-route failure, so the regressions constrain the probe rather than a test-local replica.Adversarial review notes
The late Codex review found two actionable probe/executor eligibility mismatches. This update adds red file-backed regressions for both: validation-failed rows are not pending, and previously parsed, unmaterialized CAS rows drain through the repair engine.
What this PR does not do
No production archive, daemon, migration, deployment, reprocess, dry-run apply, or receipt was mutated. The 2026-08-04 preflight observed 112 raw failures, including 111 parse failures and one maintenance failure. This PR does not remeasure that live population. Its current size and any reduction remain unknown until the open successor runs a fresh backup-gated census and reviewed apply.