Skip to content

feat(storage): append-chain backfill actuator + frontier executability lint - #3588

Merged
Sinity merged 3 commits into
masterfrom
feature/feat/raw-authority-append-chain-backfill-and-frontier-lint
Aug 2, 2026
Merged

feat(storage): append-chain backfill actuator + frontier executability lint#3588
Sinity merged 3 commits into
masterfrom
feature/feat/raw-authority-append-chain-backfill-and-frontier-lint

Conversation

@Sinity

@Sinity Sinity commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Summary

Lands items 3 and 4 of polylogue-lb39z's five-item raw-authority Phase 1 program (items 1-2 already merged via PR #3574 and #3577). Both items are low-risk (read-only classifier + dry-run actuator; static AST lint) and share the same investigation session, so they land in one PR with two focused commits rather than two separate PRs.

Problem

Item 3: 2,712 raw_sessions rows are revision_kind='append', revision_authority='quarantined', and have zero raw_session_memberships rows at all -- a genuine fixed point. The only mechanism that ever promotes an append raw (_promote_contiguous_append_evidence) requires its byte-contiguous predecessor to already be byte_proven; when the predecessor is itself stuck quarantined, no amount of re-running that cascade reaches the child.

Item 4: polylogue-w32w found RawAuthorityFrontierState.UNRESOLVED_PROVENANCE paired with the dispatched RawAuthorityActuator.REFINE_QUARANTINE -- an actuator no path (daemon or operator) could ever select. 4,174 blockers accumulated behind this for weeks undetected. PR #3466 added a runtime constructor guard (RawAuthorityFrontierItem.__post_init__) for the exact shape, but a constructor guard only fires when something actually constructs the bad pairing -- w32w's own close note explicitly left this bead's broader ask (a static lab-policy check) open.

Solution

Item 3 (polylogue/storage/raw_append_chain_backfill.py + polylogue/maintenance/raw_append_chain_backfill_apply.py): a new read-only classifier proves each membershipless append row's own claimed [append_start_offset:append_end_offset) byte range directly against its live source file's current bytes, reusing the exact byte-window comparison polylogue-u19l's live_source_reconciliation module already validated -- a proof independent of any ancestor's authority. The actuator follows the identical dry-run-default / verified-backup-required-to-apply / immutable-receipt pattern as the merged u19l and lb39z-item-2 actuators, promoting exact matches to byte_proven (reusing the existing live_source_verification_v1 evidence value -- the mechanism is identical, only the target population differs). It deliberately never touches predecessor_raw_id/baseline_raw_id/acquisition_generation; once a row is proven, the existing cascade picks it up for free on the next convergence pass. Adds migration 020 (source schema v19->v20, new raw_append_chain_backfill_receipts table) and devtools workspace raw-append-chain-backfill-apply.

Item 4 (devtools/verify_raw_authority_frontier_executability.py): statically parses polylogue/storage/raw_reconciler.py and enumerates every literal (state, actuator) pair constructible via _item(...) and _StrategyOverride(...) call sites (17 pairs on current source; 1 dynamic forwarding site correctly reported informational-only, since its literal source is separately covered). Each pair is re-checked against the real _EXECUTABLE_STATES/_APPLY_DISPATCHED_ACTUATORS imported directly from raw_reconciler.py, so this lint can never drift out of sync with the actual gate. Wired as devtools lab policy raw-authority-frontier-executability.

Not in scope / explicitly deferred

Item 5 of polylogue-lb39z (re-classify the ambiguous cohort + wire _maximal_evidence_fallback) is intentionally NOT in this PR -- flagged in the bead as the highest-risk item, directly touching the never-retire-an-accepted-head invariant. See the bead notes for a detailed account of what's needed and why it needs its own dedicated, unhurried session.

No live archive mutation was performed or attempted. Both actuators are proven only against synthetic fixtures; live application is a separate, later, operator-supervised step.

Verification

  • devtools test tests/unit/storage/test_raw_append_chain_backfill.py tests/unit/maintenance/test_raw_append_chain_backfill_apply.py -- 5 passed
  • devtools test tests/unit/storage/test_durable_migrations.py -- 40 passed (hardcoded v19->v20 migration-chain assertions updated)
  • devtools test tests/unit/devtools/test_verify_raw_authority_frontier_executability.py -- 6 passed, including an anti-vacuity test reproducing the exact pre-fix(storage): stop promising an actuator the executability gate can't select #3466 defect shape in a synthetic fixture and proving the lint catches it
  • devtools test -k raw_authority -- 90 passed, 6 failed (confirmed pre-existing/unrelated: test_raw_authority_scale_proof.py / test_raw_authority_daemon_health_proof.py, identical to the prior lb39z session's findings)
  • devtools test -k raw_materialization -- 118 passed
  • devtools verify --quick -- clean (format, lint, mypy --strict, render all --check, topology projection regenerated)

Ref polylogue-lb39z (Phase 1, items 3 and 4) / polylogue-w32w

Co-Authored-By: Claude noreply@anthropic.com

@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@Sinity, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 34 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 987a3cb4-4431-492f-958c-47337403e6a8

📥 Commits

Reviewing files that changed from the base of the PR and between 480c5a1 and 11a253e.

📒 Files selected for processing (14)
  • devtools/command_catalog.py
  • devtools/raw_append_chain_backfill_apply.py
  • devtools/verify.py
  • devtools/verify_raw_authority_frontier_executability.py
  • docs/devtools.md
  • docs/plans/topology-target.yaml
  • polylogue/maintenance/raw_append_chain_backfill_apply.py
  • polylogue/storage/raw_append_chain_backfill.py
  • polylogue/storage/sqlite/archive_tiers/source.py
  • polylogue/storage/sqlite/migrations/source/020_raw_append_chain_backfill_receipts.sql
  • tests/unit/devtools/test_verify_raw_authority_frontier_executability.py
  • tests/unit/maintenance/test_raw_append_chain_backfill_apply.py
  • tests/unit/storage/test_durable_migrations.py
  • tests/unit/storage/test_raw_append_chain_backfill.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sinity and others added 3 commits August 2, 2026 22:54
…raws

Problem: 2,712 raw_sessions rows are revision_kind='append',
revision_authority='quarantined', and have zero raw_session_memberships
rows -- a genuine fixed point. The only mechanism that ever promotes an
append raw (_promote_contiguous_append_evidence,
storage/sqlite/archive_tiers/revision_governance.py) requires its
byte-contiguous predecessor to already be revision_authority='byte_proven'.
When the predecessor itself is stuck quarantined, re-running that cascade
can never reach the child.

Solution: a new read-only classifier
(polylogue/storage/raw_append_chain_backfill.py) proves each such row's own
claimed [append_start_offset:append_end_offset) byte range directly against
its live source file's current bytes, reusing the exact byte-window
comparison polylogue-u19l's live_source_reconciliation module already
validated -- a proof that does not depend on any ancestor's authority. A
new actuator (polylogue/maintenance/raw_append_chain_backfill_apply.py)
follows the identical dry-run-default / verified-backup-required-to-apply /
immutable-receipt pattern as the merged u19l and lb39z-item-2 actuators,
promoting exact matches to revision_authority='byte_proven' (reusing the
existing 'live_source_verification_v1' evidence value; the mechanism is
identical, only the target population differs). It deliberately never
touches predecessor_raw_id/baseline_raw_id/acquisition_generation -- once
proven, the existing cascade picks a row up for free on the next
convergence pass, either resolving it as its true predecessor's child or
using it as a newly eligible parent for whatever fragment sits downstream.

Adds migration 020 (source schema v19->v20): a new
raw_append_chain_backfill_receipts table, and a new
devtools workspace raw-append-chain-backfill-apply command.

Verification: devtools test tests/unit/storage/test_raw_append_chain_backfill.py
tests/unit/maintenance/test_raw_append_chain_backfill_apply.py -- 5 passed.
devtools test tests/unit/storage/test_durable_migrations.py -- 40 passed
(hardcoded v19->v20 migration-chain assertions updated). devtools test -k
raw_authority -- 90 passed, 6 failed (confirmed pre-existing/unrelated:
test_raw_authority_scale_proof.py / test_raw_authority_daemon_health_proof.py,
identical to the prior lb39z session's findings). devtools test -k
raw_materialization -- 118 passed. mypy --strict clean.

Not run against the live archive -- that is a separate, later,
operator-supervised step, identical to every prior actuator in this
program.

Ref polylogue-lb39z (Phase 1, item 3)

Co-Authored-By: Claude <noreply@anthropic.com>
Problem: polylogue-w32w found RawAuthorityFrontierState.UNRESOLVED_PROVENANCE
paired with the dispatched RawAuthorityActuator.REFINE_QUARANTINE -- an
actuator no path (daemon or operator break-glass) could ever select, since
only states in _EXECUTABLE_STATES are ever chosen. 4,174 blockers
accumulated behind this unreachable actuator for weeks before anyone
noticed. PR #3466 fixed the specific defect and added a constructor-level
invariant (RawAuthorityFrontierItem.__post_init__ raises for this exact
shape) plus a drift-guard test tying _APPLY_DISPATCHED_ACTUATORS to
apply()'s real dispatch branches. That closed polylogue-w32w, but its
close note explicitly left this bead's broader ask open: a devtools
lab-policy check, not just a constructor guard, so a FUTURE unreachable
pairing fails at review time even for a branch no test happens to
exercise (a runtime constructor guard only fires when something actually
constructs the bad combination).

Solution: devtools/verify_raw_authority_frontier_executability.py
statically parses polylogue/storage/raw_reconciler.py and enumerates
every literal (state, actuator) pair constructible via _item(...) and
_StrategyOverride(...) call sites (17 pairs found on the current source;
1 dynamic forwarding site correctly reported as informational-only,
since its underlying literal source is already covered by its
_StrategyOverride construction site). Each pair is re-checked against
the real _EXECUTABLE_STATES / _APPLY_DISPATCHED_ACTUATORS imported
directly from raw_reconciler.py (never re-declared), so this lint can
never drift out of sync with the actual executability gate. Wired as
`devtools lab policy raw-authority-frontier-executability`.

Verification: devtools test tests/unit/devtools/test_verify_raw_authority_frontier_executability.py
-- 6 passed, including an anti-vacuity test reproducing the exact
pre-#3466 UNRESOLVED_PROVENANCE+REFINE_QUARANTINE shape in a synthetic
fixture module and proving the lint flags it, plus controls proving a
safe pairing, a RawAuthorityActuator.NONE pairing, a dynamic-site
pairing, and an unknown-enum-member typo all behave correctly.
devtools lab policy raw-authority-frontier-executability against the
live repo: 0 violations. devtools verify --quick clean (ruff
format/check, mypy --strict, render all --check, topology projection
regenerated for the new module).

Ref polylogue-lb39z (Phase 1, item 4) / polylogue-w32w

Co-Authored-By: Claude <noreply@anthropic.com>
…ck gate

Landed in 83e3fb0 as a registered devtools lab-policy command but never
added to build_verify_steps, so it never ran in devtools verify --quick/CI
despite being static, archive-independent, and sub-second like its three
siblings (demo-tour-freshness, raw-payload-hash-purity,
position-derived-identity) immediately above it in the same block.
@Sinity
Sinity force-pushed the feature/feat/raw-authority-append-chain-backfill-and-frontier-lint branch from 83e3fb0 to 11a253e Compare August 2, 2026 21:01
@Sinity
Sinity merged commit f7dfef4 into master Aug 2, 2026
3 checks passed
@Sinity
Sinity deleted the feature/feat/raw-authority-append-chain-backfill-and-frontier-lint branch August 2, 2026 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant