feat(archive): wire guarded presence-guarantee fallback for headless membership conflicts - #3630
Conversation
…membership conflicts Problem: classify_membership_revisions quarantined every genuine, irreducible membership conflict with nothing accepted, even when the logical source had never had ANY head materialized before -- a document could vanish from the archive permanently just because two revisions couldn't be ordered by containment. _maximal_evidence_fallback existed, was unit-tested, but was deliberately never called (polylogue-lb39z item 5): wiring it in unconditionally would let a later membership pass silently retire an already-accepted head in favor of an unrelated raw (polylogue-miwv, PR #3211's write-back invariant), proven by two real integration-test regressions during earlier work on this bead. Solution: classify_membership_revisions gains an existing_accepted_raw_id keyword (default None). The fallback applies ONLY when no head exists yet for the cohort under ANY authority. This guard is narrower than "only refuse a different raw_id": a second regression surfaced during development showed that re-accepting the SAME raw_id through membership governance still overwrites a byte-governed head's own accepted_frontier_kind/generation metadata (byte -> semantic), a real authority downgrade despite the pointed-to raw_id never changing (test_live_multi_session_divergence_reopens_raw_authority). Both production call sites (polylogue/sources/live/batch.py's incremental ingest, polylogue/sources/revision_backfill.py's offline rebuild) now pass the cohort's current accepted head raw_id (already computed at both sites) through to the classifier. Updated ~15 existing unit/integration tests whose fixtures were genuinely headless conflicts (no prior accepted head) to reflect the new presence-guarantee resolution instead of asserting quarantine; added explicit guard tests for the three cases (no prior head applies fallback, prior head at a different raw_id refuses, prior head at the SAME raw_id as the fallback's own pick still refuses). Deferred: item 5(a) of polylogue-lb39z (attachment-id-stability normalizer) was already fully closed per prior session notes, confirmed unchanged. Item 4 (frontier-executability lint) and item 3 (append-chain backfill) were completed in prior sessions on this bead. This PR is the remainder of item 5(b)/(c). Verification: - devtools test tests/unit/archive/test_session_revision_membership.py tests/unit/sources/test_revision_backfill.py tests/unit/sources/test_live_batch_support.py tests/unit/storage/test_revision_replay.py tests/unit/pipeline/test_ingest_batch.py tests/unit/sources/test_parsers_drive.py tests/unit/sources/test_live_watcher.py -> "4 failed, 393 passed" -- the 4 failures reproduce identically with this change reverted (git stash), confirmed pre-existing/unrelated (test_full_ingest_writes_archive_with_route_observability, test_full_ingest_skips_durably_excised_content_without_aborting_batch, test_append_multi_session_payload_is_rejected_before_index_write, test_backfill_content_cache_across_pages_reduces_parses_and_matches_uncached_archive). - devtools verify --quick -> exit_code 0 (format/lint/mypy/render all/ layering/schema-versioning/raw-authority-frontier-executability all ok). Ref polylogue-lb39z
|
Warning Review limit reached
Next review available in: 45 seconds Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
test_live_batch_support.py: normal.jsonl fixture used bare '{}' which
polylogue-lb39z's guarded presence-guarantee fallback (#3630) now
correctly refuses on real re-parse -- give it a real minimal session.
test_live_watcher.py: os.cpu_count moved from ingest_batch._core to
process_pool.py during today's refactors; 2 monkeypatch targets updated.
Co-Authored-By: Claude <noreply@anthropic.com>
Summary
Wires
_maximal_evidence_fallbackintoclassify_membership_revisionsfor genuine, irreducible membership conflicts, guarded so it only fires when a logical source has never had ANY accepted head. This is polylogue-lb39z item 5(b)/(c), the last remaining item of the "Raw-authority redesign Phase 1: drain the fake quarantine" bead.Problem
classify_membership_revisionsquarantined every genuine, irreducible membership conflict withaccepted_raw_ids == (), even when the logical source had never had any head materialized at all — a document could vanish from the archive permanently just because two revisions couldn't be ordered by containment._maximal_evidence_fallback(deterministic maximal-evidence pick, order-independent, unit-tested) existed for exactly this but was deliberately never called: wiring it in unconditionally would let a later membership pass silently retire an already-accepted head in favor of an unrelated raw, tripping a real, carefully-designedapply_raw_membership_classificationwrite-back invariant (polylogue-miwv, PR #3211).Solution
classify_membership_revisionsgains anexisting_accepted_raw_id: str | None = Nonekeyword. The fallback applies ONLY whenexisting_accepted_raw_id is None— no head exists yet for the cohort under ANY authority (byte-governed or membership-governed). Both production call sites now pass the cohort's current accepted head raw_id through:polylogue/sources/live/batch.py(incremental live ingest) — already computedaccepted_head_raw_idunconditionally at the call site, just wired the pass-through.polylogue/sources/revision_backfill.py(offline rebuild) —head_raw_idwas already fetched unconditionally but only conditionally absorbed into the candidate cohort when quarantined; the pass-through to the classifier is now unconditional so the guard also knows about a chain-governed (non-quarantined) existing head it deliberately never absorbs into the comparison.Non-obvious decision, found during verification, not in the original plan: the guard is deliberately narrower than "only refuse when the fallback would pick a different raw_id than the existing head." A "pure re-affirmation" (fallback pick == existing head's raw_id) is NOT actually safe:
apply_raw_membership_classificationre-accepting that exact raw_id through membership governance still overwrites the head's ownaccepted_frontier_kind/generation metadata (e.g. downgrading a byte-governed head to"semantic", generation 17 → 2), even though the pointed-to raw_id never changed. This was caught by a real integration-test regression (test_live_multi_session_divergence_reopens_raw_authority) during development, not a theoretical concern, so the guard refuses ANY existing head, not just a mismatched one.Updated ~15 existing unit/integration test assertions whose fixtures were genuinely-headless conflicts (no prior accepted head) to reflect the new presence-guarantee resolution instead of asserting quarantine, with inline comments computing the expected fallback pick. Added three explicit guard tests covering: no prior head (fallback applies), prior head at a different raw_id (refuses), and prior head at the SAME raw_id as the fallback's own pick (still refuses, per the finding above).
Verification
→
4 failed, 393 passed— the 4 failures (test_full_ingest_writes_archive_with_route_observability,test_full_ingest_skips_durably_excised_content_without_aborting_batch,test_append_multi_session_payload_is_rejected_before_index_write,test_backfill_content_cache_across_pages_reduces_parses_and_matches_uncached_archive) reproduce identically with this change reverted (git stash), confirmed pre-existing/unrelated.→
"exit_code": 0(format/lint/mypy/render all/layering/schema-versioning/raw-authority-frontier-executability all ok).Out of scope / deferred
logical_source_keyat all) and is not touched by this PR.Ref polylogue-lb39z