fix(storage): exclude byte-identical duplicates from revision baseline tie-break - #3616
Conversation
…e tie-break Problem PR #3574 collapsed byte-identical full-revision captures into one representative plus "duplicate" decisions, and a follow-up fix (#3580) made a duplicate's acquisition_generation mirror its representative's generation for correct display ordering. Together these made a duplicate of the accepted baseline/head share that node's generation number. plan_revision_replay (polylogue/archive/revision_replay.py) picks the unique baseline by finding the single candidate with the newest acquisition_generation among FULL, byte-proven candidates; a duplicate sharing that generation now looked like a second competing baseline, so the tie-break declared the cohort ambiguous and returned an empty accepted_raw_ids for a cohort that in fact has one unambiguous baseline. Backfill and rebuild callers (sources/revision_backfill.py) treat an empty accepted_raw_ids as "no accepted chain" and fold every full-only raw for the identity into membership governance via replace_raw_membership_census(..., retire_full_revision_governance= True). That path's pre-existing (#3406) guard then raised ActiveByteRevisionChainError the moment it tried to retire the baseline raw, because the duplicate's own baseline_raw_id column still durably points at it. Solution The guard itself is correct and intentionally strict: a raw genuinely still pointed at by another raw's predecessor_raw_id/baseline_raw_id must not be retired out of byte governance. The defect is upstream, in plan_revision_replay's inability to distinguish a harmless duplicate from a genuinely competing baseline. A FULL, byte-proven candidate with no predecessor_raw_id that is NOT itself the cohort's baseline_raw_id can only be such a duplicate -- the classifier that writes these columns (archive/revision_authority.py) guarantees at most one true root per cohort. plan_revision_replay now excludes that signature from the baseline candidate pool; excluded duplicates fall through to the existing trailing loop and are marked DEFERRED against whatever head the genuine evidence accepts. Verification - devtools test tests/unit/sources/test_revision_backfill.py::test_backfill_content_cache_across_pages_reduces_parses_and_matches_uncached_archive tests/unit/storage/test_rebuild_paging_content_order.py::test_rebuild_content_order_paging_dedups_first_time_classification_via_content_cache -- 2 passed (previously failing with ActiveByteRevisionChainError). - devtools test tests/unit/storage/test_revision_replay.py tests/unit/storage/test_raw_revision_authority.py tests/unit/sources/test_revision_backfill.py tests/unit/storage/test_rebuild_paging_content_order.py tests/unit/sources/test_live_batch_support.py -k "revision or membership or duplicate or backfill or paging" -- 127 passed. - devtools test tests/unit/storage/test_revision_replay.py tests/unit/sources/test_revision_backfill.py tests/unit/storage/test_rebuild_paging_content_order.py tests/unit/sources/test_live_batch_support.py (full files) -- 169 passed, 3 pre-existing failures in test_live_batch_support.py confirmed identical with this change reverted (git stash), unrelated to this fix. - Red-first: new tests confirmed failing against the unpatched revision_replay.py (git stash), then passing after the fix. - devtools verify --quick running in background; result to be added before PR/merge. Ref polylogue-qhk8z
|
Warning Review limit reached
Next review available in: 49 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
6753s (byte-dup supersession, PR #3624), c831 (message_type restamp, PR #3629, live backfill verified 0 remaining), qhk8z (duplicate-chain baseline tie-break, PR #3616), ct3r2 + t6iga (merge-gate enforcement, PR #3607, duplicate beads for the same fix). Co-Authored-By: Claude <noreply@anthropic.com>
Summary
Fixes a cross-feature interaction between PR #3574 (byte-identical duplicate collapse) and the pre-existing (#3406) membership-census guard: a duplicate of the accepted revision baseline could make
plan_revision_replaymisclassify a cohort with one unambiguous baseline as ambiguous, which then routed backfill/rebuild into a fallback path that trippedActiveByteRevisionChainError.Problem
PR #3574 collapses byte-identical full-revision captures into one representative plus "duplicate" decisions. A follow-up fix (#3580) made a duplicate's
acquisition_generationmirror its representative's generation for correct display ordering. Together these made a duplicate of the accepted baseline/head share that node's generation number.plan_revision_replay(polylogue/archive/revision_replay.py) selects the unique baseline by finding the single FULL, byte-proven candidate with the newestacquisition_generation. A duplicate sharing that generation now looked like a second competing baseline, so the tie-break declared the cohort ambiguous and returned an emptyaccepted_raw_idsfor a cohort that in fact has one unambiguous baseline.Backfill/rebuild callers (
sources/revision_backfill.py) treat an emptyaccepted_raw_idsas "no accepted chain" and fold every full-only raw for the identity into membership governance viareplace_raw_membership_census(..., retire_full_revision_governance=True). That path's guard then raisedActiveByteRevisionChainErrorthe moment it tried to retire the baseline raw, because the duplicate's ownbaseline_raw_idcolumn still durably points at it.Confirmed via
git log -S "ActiveByteRevisionChainError"that the guard itself is unchanged since #3406; the trigger is #3574/#3580's newly-created generation-sharing shape, not the guard.Solution
The guard is correct and intentionally strict: a raw genuinely still pointed at by another raw's
predecessor_raw_id/baseline_raw_idmust not be retired out of byte governance (a real dependent chain shouldn't lose its baseline out from under it). The defect is upstream, inplan_revision_replay's inability to distinguish a harmless duplicate from a genuinely competing baseline.A FULL, byte-proven candidate with no
predecessor_raw_idthat is NOT itself the cohort'sbaseline_raw_idcan only be such a duplicate -- the classifier that writes these columns (archive/revision_authority.py) guarantees at most one true root per cohort (relation="baseline").plan_revision_replaynow excludes that signature from the baseline candidate pool via_is_full_duplicate_signature; excluded duplicates still fall through to the existing trailing loop and are markedDEFERREDagainst whatever head the genuine evidence accepts.No schema change; this is a pure classification/replay-planning fix in
polylogue/archive/revision_replay.py, shared correctly by both the backfill/rebuild path (revision_governance.classify_raw_revision_cohort) and the live watcher (sources/live/batch.py), which both buildRevisionCandidatelists from the sameraw_sessionscolumns.Verification
devtools test tests/unit/sources/test_revision_backfill.py::test_backfill_content_cache_across_pages_reduces_parses_and_matches_uncached_archive tests/unit/storage/test_rebuild_paging_content_order.py::test_rebuild_content_order_paging_dedups_first_time_classification_via_content_cache-- 2 passed (previously failing withActiveByteRevisionChainError; this is the reproduction the tracking item names).devtools test tests/unit/storage/test_revision_replay.py tests/unit/storage/test_raw_revision_authority.py tests/unit/sources/test_revision_backfill.py tests/unit/storage/test_rebuild_paging_content_order.py tests/unit/sources/test_live_batch_support.py -k "revision or membership or duplicate or backfill or paging"-- 127 passed.test_live_batch_support.pyconfirmed identical with this change reverted (git stash); unrelated to this fix.test_revision_replay.pywere confirmed failing against the unpatchedrevision_replay.py(git stash), then passing after the fix.devtools verify --quick-- exit 0 (format, lint, mypy --strict, render all --check, layering, closure-matrix, schema roundtrip, schema-versioning, classifier-fingerprints, schema-promotion-audit, etc., all passed).Anti-vacuity: the failing production path exercised is
sources/revision_backfill.py'sif not plan.accepted_raw_ids:fallback intoarchive.replace_raw_membership_census(..., retire_full_revision_governance=True); the mutation that makes the new tests fail is reverting the_is_full_duplicate_signatureexclusion inplan_revision_replay(confirmed viagit stashonpolylogue/archive/revision_replay.pyalone). The guard invariant itself (a real dependent chain must block retirement) is separately re-asserted inside the new archive-level test by callingreplace_raw_membership_censusdirectly and expectingActiveByteRevisionChainError, and is unaffected by this change (pre-existingtest_membership_sweep_defers_sibling_retirement_instead_of_quarantining_current_rawintest_live_batch_support.pystill passes unmodified).Ref polylogue-qhk8z
Scope not addressed
None -- this closes the full interaction described in the tracking item (root-cause fix in the replay planner, not a guard loosening), with a regression test reproducing the exact reported failure plus a unit-level test pinning the underlying tie-break defect.