fix(storage): copy duplicate revisions' generation from their representative - #3580
Conversation
|
Warning Review limit reached
Next review available in: 44 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ntative Problem: _expand_duplicate_decisions (archive/revision_authority.py) gives every duplicate member predecessor_raw_id=None, so classify_raw_revision_cohort's generation walk (a dict comprehension keyed by predecessor_raw_id) never sees duplicates and they fall back to acquisition_generation=0 regardless of true chain position. Flagged by CodeRabbit review on PR #3574 (polylogue-5unky); the obvious fix (mirror the representative's predecessor_raw_id onto the duplicate) was rejected because it makes the duplicate and its representative compete for the same predecessor-keyed dict slot, risking silently dropping the real chain-continuing representative from the walk. What changed: added HistoricalRevisionDecision.duplicate_of_raw_id, populated only for relation="duplicate" entries, deliberately kept separate from predecessor_raw_id so a duplicate never enters the predecessor-keyed walk. classify_raw_revision_cohort now runs a separate post-pass after the real chain walk that copies each duplicate's generation directly from its already-computed representative. Verification: devtools test tests/unit/storage/test_revision_replay.py -k duplicate (2 passed, both proven to fail against the pre-fix code); devtools test -k raw_authority (90 passed, 6 pre-existing/unrelated scale-proof timeouts confirmed by rerunning that file with this change stashed out, same failure set); devtools verify --quick (exit_code 0, 19/19 steps green). Ref polylogue-5unky Co-Authored-By: Claude <noreply@anthropic.com>
Ref polylogue-5unky Co-Authored-By: Claude <noreply@anthropic.com>
09abcc3 to
4f98208
Compare
…e tie-break (#3616) ## Summary Fixes a cross-feature interaction between PR #3574 (byte-identical duplicate collapse) and the pre-existing (#3406) membership-census guard: a duplicate of the accepted revision baseline could make `plan_revision_replay` misclassify a cohort with one unambiguous baseline as ambiguous, which then routed backfill/rebuild into a fallback path that tripped `ActiveByteRevisionChainError`. ## Problem PR #3574 collapses byte-identical full-revision captures into one representative plus "duplicate" decisions. A follow-up fix (#3580) made a duplicate's `acquisition_generation` mirror its representative's generation for correct display ordering. Together these made a duplicate of the accepted baseline/head share that node's generation number. `plan_revision_replay` (`polylogue/archive/revision_replay.py`) selects the unique baseline by finding the single FULL, byte-proven candidate with the newest `acquisition_generation`. A duplicate sharing that generation now looked like a second competing baseline, so the tie-break declared the cohort ambiguous and returned an empty `accepted_raw_ids` for a cohort that in fact has one unambiguous baseline. Backfill/rebuild callers (`sources/revision_backfill.py`) treat an empty `accepted_raw_ids` as "no accepted chain" and fold every full-only raw for the identity into membership governance via `replace_raw_membership_census(..., retire_full_revision_governance=True)`. That path's guard then raised `ActiveByteRevisionChainError` the moment it tried to retire the baseline raw, because the duplicate's own `baseline_raw_id` column still durably points at it. Confirmed via `git log -S "ActiveByteRevisionChainError"` that the guard itself is unchanged since #3406; the trigger is #3574/#3580's newly-created generation-sharing shape, not the guard. ## Solution The guard is correct and intentionally strict: a raw genuinely still pointed at by another raw's `predecessor_raw_id`/`baseline_raw_id` must not be retired out of byte governance (a real dependent chain shouldn't lose its baseline out from under it). The defect is upstream, in `plan_revision_replay`'s inability to distinguish a harmless duplicate from a genuinely competing baseline. A FULL, byte-proven candidate with no `predecessor_raw_id` that is NOT itself the cohort's `baseline_raw_id` can only be such a duplicate -- the classifier that writes these columns (`archive/revision_authority.py`) guarantees at most one true root per cohort (`relation="baseline"`). `plan_revision_replay` now excludes that signature from the baseline candidate pool via `_is_full_duplicate_signature`; excluded duplicates still fall through to the existing trailing loop and are marked `DEFERRED` against whatever head the genuine evidence accepts. No schema change; this is a pure classification/replay-planning fix in `polylogue/archive/revision_replay.py`, shared correctly by both the backfill/rebuild path (`revision_governance.classify_raw_revision_cohort`) and the live watcher (`sources/live/batch.py`), which both build `RevisionCandidate` lists from the same `raw_sessions` columns. ## Verification - `devtools test tests/unit/sources/test_revision_backfill.py::test_backfill_content_cache_across_pages_reduces_parses_and_matches_uncached_archive tests/unit/storage/test_rebuild_paging_content_order.py::test_rebuild_content_order_paging_dedups_first_time_classification_via_content_cache` -- 2 passed (previously failing with `ActiveByteRevisionChainError`; this is the reproduction the tracking item names). - `devtools test tests/unit/storage/test_revision_replay.py tests/unit/storage/test_raw_revision_authority.py tests/unit/sources/test_revision_backfill.py tests/unit/storage/test_rebuild_paging_content_order.py tests/unit/sources/test_live_batch_support.py -k "revision or membership or duplicate or backfill or paging"` -- 127 passed. - Full-file runs of the same five files -- 169 passed, 3 pre-existing failures in `test_live_batch_support.py` confirmed identical with this change reverted (`git stash`); unrelated to this fix. - Red-first: the two new tests in `test_revision_replay.py` were confirmed failing against the unpatched `revision_replay.py` (`git stash`), then passing after the fix. - `devtools verify --quick` -- exit 0 (format, lint, mypy --strict, render all --check, layering, closure-matrix, schema roundtrip, schema-versioning, classifier-fingerprints, schema-promotion-audit, etc., all passed). Anti-vacuity: the failing production path exercised is `sources/revision_backfill.py`'s `if not plan.accepted_raw_ids:` fallback into `archive.replace_raw_membership_census(..., retire_full_revision_governance=True)`; the mutation that makes the new tests fail is reverting the `_is_full_duplicate_signature` exclusion in `plan_revision_replay` (confirmed via `git stash` on `polylogue/archive/revision_replay.py` alone). The guard invariant itself (a real dependent chain must block retirement) is separately re-asserted inside the new archive-level test by calling `replace_raw_membership_census` directly and expecting `ActiveByteRevisionChainError`, and is unaffected by this change (pre-existing `test_membership_sweep_defers_sibling_retirement_instead_of_quarantining_current_raw` in `test_live_batch_support.py` still passes unmodified). Ref polylogue-qhk8z ## Scope not addressed None -- this closes the full interaction described in the tracking item (root-cause fix in the replay planner, not a guard loosening), with a regression test reproducing the exact reported failure plus a unit-level test pinning the underlying tie-break defect.
Summary
Duplicate revision decisions (byte-identical to another raw in the same
cohort) now record their true
acquisition_generation(copied from theirrepresentative) instead of always falling back to 0.
Problem
_expand_duplicate_decisions(polylogue/archive/revision_authority.py)gives every duplicate member
predecessor_raw_id=None.revision_governance .py's generation walk inclassify_raw_revision_cohortbuilds apredecessor-keyed dict (
{decision.predecessor_raw_id: decision.raw_id for decision in decisions if decision.predecessor_raw_id is not None}) andwalks it from the baseline. Because duplicates never carry a
predecessor_raw_id, they never appear in that dict and never get visitedby the walk, so they fall back to
acquisition_generation=0regardless oftheir representative's true chain position. Flagged by CodeRabbit review on
PR #3574 (polylogue-5unky).
The obvious fix — mirror the representative's
predecessor_raw_idonto theduplicate — was considered and rejected: it makes the duplicate and its
representative compete for the same key in the plain-dict comprehension
above. Whichever entry the comprehension writes last silently wins that
slot, so the duplicate can overwrite the real chain-continuing
representative and break generation numbering for everything downstream of
it. That would be a worse regression than the bug being fixed.
Severity is low:
acquisition_generationon a duplicate row is adisplay/ordering field, not something authority/replay-plan logic depends
on for duplicates. Still worth fixing correctly since this is a
safety-adjacent subsystem.
Solution
Implemented approach (b) from the two options the tracking bead named: a
separate post-pass that copies the representative's already-computed
generation, rather than folding duplicates into the predecessor-keyed walk
at all.
HistoricalRevisionDecision.duplicate_of_raw_id: str | None(
polylogue/archive/revision_authority.py), populated only forrelation="duplicate"entries in_expand_duplicate_decisions, pointingat the representative raw_id. Deliberately kept separate from
predecessor_raw_id-- a duplicate is not a chain-continuing child ofanything, it's a second copy of its representative's own bytes, so it
must never compete for a
childrendict slot.classify_raw_revision_cohort(
polylogue/storage/sqlite/archive_tiers/revision_governance.py) now runsa post-pass after the real chain walk: for every
relation="duplicate"decision,
generation_by_raw_id[dup.raw_id] = generation_by_raw_id.get(dup.duplicate_of_raw_id, 0). Duplicates neverparticipate in the predecessor-keyed walk.
Approach (a) (children-as-list, picking the chain-continuing child
specifically) was considered but doesn't fit the semantics as cleanly: a
duplicate isn't an additional child of the representative's predecessor,
it's the same generation as the representative itself.
Verification
devtools test tests/unit/storage/test_revision_replay.py -k duplicate--2 passed. Both new tests were confirmed to fail against the pre-fix code
(
assert 0 == 1) by stashing the fix and rerunning, proving they exercisethe real bug, then re-verified passing with the fix restored.
test_duplicate_decision_mid_chain_gets_representative_generation_not_zero:builds a 3-link byte chain (base -> mid -> head) plus a byte-identical
duplicate of the middle link, proves the duplicate gets generation 1
(mid's real position), not the 0 fallback.
test_duplicate_generation_copy_does_not_drop_the_chain_continuing_representative:constructs the exact collision shape the rejected fix would have hit --
a duplicate of
midthat is same-size and lexicographically laterthan
mid(the ordering the rejected fix's dict comprehension wouldneed to let the duplicate clobber
mid's real predecessor-keyed slot)-- and proves
head, two links downstream ofmid, still gets itscorrect real generation (2), i.e. the true chain-continuing
representative was never displaced from the walk.
devtools test -k raw_authority-- 90 passed, 6 failed. All 6 failuresare in
test_raw_authority_scale_proof.py/test_raw_authority_daemon_health_proof.py(timeout-driven scale/daemontests). Confirmed pre-existing and unrelated: reran
test_raw_authority_scale_proof.pyalone with this change stashed out andgot the identical failure set.
devtools verify --quick--exit_code: 0, all 19 steps green (ruffformat/check, mypy, render all, topology, layering, closure-matrix, schema
roundtrip, manifests, ci-workflows, doc-commands, docs-coverage,
test-infra-currency, pytest-timeout-overrides, degrade-loudly,
hash-boundary-census, schema-versioning policy, classifier-fingerprints
policy, schema promotion audit). Also ran again automatically via the
pre-push hook -- green.
Ref polylogue-5unky