feat(storage): append-chain backfill actuator + frontier executability lint - #3588
Conversation
|
Warning Review limit reached
Next review available in: 34 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (14)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…raws Problem: 2,712 raw_sessions rows are revision_kind='append', revision_authority='quarantined', and have zero raw_session_memberships rows -- a genuine fixed point. The only mechanism that ever promotes an append raw (_promote_contiguous_append_evidence, storage/sqlite/archive_tiers/revision_governance.py) requires its byte-contiguous predecessor to already be revision_authority='byte_proven'. When the predecessor itself is stuck quarantined, re-running that cascade can never reach the child. Solution: a new read-only classifier (polylogue/storage/raw_append_chain_backfill.py) proves each such row's own claimed [append_start_offset:append_end_offset) byte range directly against its live source file's current bytes, reusing the exact byte-window comparison polylogue-u19l's live_source_reconciliation module already validated -- a proof that does not depend on any ancestor's authority. A new actuator (polylogue/maintenance/raw_append_chain_backfill_apply.py) follows the identical dry-run-default / verified-backup-required-to-apply / immutable-receipt pattern as the merged u19l and lb39z-item-2 actuators, promoting exact matches to revision_authority='byte_proven' (reusing the existing 'live_source_verification_v1' evidence value; the mechanism is identical, only the target population differs). It deliberately never touches predecessor_raw_id/baseline_raw_id/acquisition_generation -- once proven, the existing cascade picks a row up for free on the next convergence pass, either resolving it as its true predecessor's child or using it as a newly eligible parent for whatever fragment sits downstream. Adds migration 020 (source schema v19->v20): a new raw_append_chain_backfill_receipts table, and a new devtools workspace raw-append-chain-backfill-apply command. Verification: devtools test tests/unit/storage/test_raw_append_chain_backfill.py tests/unit/maintenance/test_raw_append_chain_backfill_apply.py -- 5 passed. devtools test tests/unit/storage/test_durable_migrations.py -- 40 passed (hardcoded v19->v20 migration-chain assertions updated). devtools test -k raw_authority -- 90 passed, 6 failed (confirmed pre-existing/unrelated: test_raw_authority_scale_proof.py / test_raw_authority_daemon_health_proof.py, identical to the prior lb39z session's findings). devtools test -k raw_materialization -- 118 passed. mypy --strict clean. Not run against the live archive -- that is a separate, later, operator-supervised step, identical to every prior actuator in this program. Ref polylogue-lb39z (Phase 1, item 3) Co-Authored-By: Claude <noreply@anthropic.com>
Problem: polylogue-w32w found RawAuthorityFrontierState.UNRESOLVED_PROVENANCE paired with the dispatched RawAuthorityActuator.REFINE_QUARANTINE -- an actuator no path (daemon or operator break-glass) could ever select, since only states in _EXECUTABLE_STATES are ever chosen. 4,174 blockers accumulated behind this unreachable actuator for weeks before anyone noticed. PR #3466 fixed the specific defect and added a constructor-level invariant (RawAuthorityFrontierItem.__post_init__ raises for this exact shape) plus a drift-guard test tying _APPLY_DISPATCHED_ACTUATORS to apply()'s real dispatch branches. That closed polylogue-w32w, but its close note explicitly left this bead's broader ask open: a devtools lab-policy check, not just a constructor guard, so a FUTURE unreachable pairing fails at review time even for a branch no test happens to exercise (a runtime constructor guard only fires when something actually constructs the bad combination). Solution: devtools/verify_raw_authority_frontier_executability.py statically parses polylogue/storage/raw_reconciler.py and enumerates every literal (state, actuator) pair constructible via _item(...) and _StrategyOverride(...) call sites (17 pairs found on the current source; 1 dynamic forwarding site correctly reported as informational-only, since its underlying literal source is already covered by its _StrategyOverride construction site). Each pair is re-checked against the real _EXECUTABLE_STATES / _APPLY_DISPATCHED_ACTUATORS imported directly from raw_reconciler.py (never re-declared), so this lint can never drift out of sync with the actual executability gate. Wired as `devtools lab policy raw-authority-frontier-executability`. Verification: devtools test tests/unit/devtools/test_verify_raw_authority_frontier_executability.py -- 6 passed, including an anti-vacuity test reproducing the exact pre-#3466 UNRESOLVED_PROVENANCE+REFINE_QUARANTINE shape in a synthetic fixture module and proving the lint flags it, plus controls proving a safe pairing, a RawAuthorityActuator.NONE pairing, a dynamic-site pairing, and an unknown-enum-member typo all behave correctly. devtools lab policy raw-authority-frontier-executability against the live repo: 0 violations. devtools verify --quick clean (ruff format/check, mypy --strict, render all --check, topology projection regenerated for the new module). Ref polylogue-lb39z (Phase 1, item 4) / polylogue-w32w Co-Authored-By: Claude <noreply@anthropic.com>
…ck gate Landed in 83e3fb0 as a registered devtools lab-policy command but never added to build_verify_steps, so it never ran in devtools verify --quick/CI despite being static, archive-independent, and sub-second like its three siblings (demo-tour-freshness, raw-payload-hash-purity, position-derived-identity) immediately above it in the same block.
83e3fb0 to
11a253e
Compare
Summary
Lands items 3 and 4 of polylogue-lb39z's five-item raw-authority Phase 1 program (items 1-2 already merged via PR #3574 and #3577). Both items are low-risk (read-only classifier + dry-run actuator; static AST lint) and share the same investigation session, so they land in one PR with two focused commits rather than two separate PRs.
Problem
Item 3: 2,712
raw_sessionsrows arerevision_kind='append',revision_authority='quarantined', and have zeroraw_session_membershipsrows at all -- a genuine fixed point. The only mechanism that ever promotes an append raw (_promote_contiguous_append_evidence) requires its byte-contiguous predecessor to already bebyte_proven; when the predecessor is itself stuck quarantined, no amount of re-running that cascade reaches the child.Item 4: polylogue-w32w found
RawAuthorityFrontierState.UNRESOLVED_PROVENANCEpaired with the dispatchedRawAuthorityActuator.REFINE_QUARANTINE-- an actuator no path (daemon or operator) could ever select. 4,174 blockers accumulated behind this for weeks undetected. PR #3466 added a runtime constructor guard (RawAuthorityFrontierItem.__post_init__) for the exact shape, but a constructor guard only fires when something actually constructs the bad pairing -- w32w's own close note explicitly left this bead's broader ask (a static lab-policy check) open.Solution
Item 3 (
polylogue/storage/raw_append_chain_backfill.py+polylogue/maintenance/raw_append_chain_backfill_apply.py): a new read-only classifier proves each membershipless append row's own claimed[append_start_offset:append_end_offset)byte range directly against its live source file's current bytes, reusing the exact byte-window comparison polylogue-u19l'slive_source_reconciliationmodule already validated -- a proof independent of any ancestor's authority. The actuator follows the identical dry-run-default / verified-backup-required-to-apply / immutable-receipt pattern as the merged u19l and lb39z-item-2 actuators, promoting exact matches tobyte_proven(reusing the existinglive_source_verification_v1evidence value -- the mechanism is identical, only the target population differs). It deliberately never touchespredecessor_raw_id/baseline_raw_id/acquisition_generation; once a row is proven, the existing cascade picks it up for free on the next convergence pass. Adds migration 020 (source schema v19->v20, newraw_append_chain_backfill_receiptstable) anddevtools workspace raw-append-chain-backfill-apply.Item 4 (
devtools/verify_raw_authority_frontier_executability.py): statically parsespolylogue/storage/raw_reconciler.pyand enumerates every literal(state, actuator)pair constructible via_item(...)and_StrategyOverride(...)call sites (17 pairs on current source; 1 dynamic forwarding site correctly reported informational-only, since its literal source is separately covered). Each pair is re-checked against the real_EXECUTABLE_STATES/_APPLY_DISPATCHED_ACTUATORSimported directly fromraw_reconciler.py, so this lint can never drift out of sync with the actual gate. Wired asdevtools lab policy raw-authority-frontier-executability.Not in scope / explicitly deferred
Item 5 of polylogue-lb39z (re-classify the ambiguous cohort + wire
_maximal_evidence_fallback) is intentionally NOT in this PR -- flagged in the bead as the highest-risk item, directly touching the never-retire-an-accepted-head invariant. See the bead notes for a detailed account of what's needed and why it needs its own dedicated, unhurried session.No live archive mutation was performed or attempted. Both actuators are proven only against synthetic fixtures; live application is a separate, later, operator-supervised step.
Verification
devtools test tests/unit/storage/test_raw_append_chain_backfill.py tests/unit/maintenance/test_raw_append_chain_backfill_apply.py-- 5 passeddevtools test tests/unit/storage/test_durable_migrations.py-- 40 passed (hardcoded v19->v20 migration-chain assertions updated)devtools test tests/unit/devtools/test_verify_raw_authority_frontier_executability.py-- 6 passed, including an anti-vacuity test reproducing the exact pre-fix(storage): stop promising an actuator the executability gate can't select #3466 defect shape in a synthetic fixture and proving the lint catches itdevtools test -k raw_authority-- 90 passed, 6 failed (confirmed pre-existing/unrelated:test_raw_authority_scale_proof.py/test_raw_authority_daemon_health_proof.py, identical to the prior lb39z session's findings)devtools test -k raw_materialization-- 118 passeddevtools verify --quick-- clean (format, lint, mypy --strict, render all --check, topology projection regenerated)Ref polylogue-lb39z (Phase 1, items 3 and 4) / polylogue-w32w
Co-Authored-By: Claude noreply@anthropic.com