Skip to content

feat(archive): wire guarded presence-guarantee fallback for headless membership conflicts - #3630

Merged
Sinity merged 1 commit into
masterfrom
feature/fix/wire-membership-presence-guarantee-fallback
Aug 3, 2026
Merged

feat(archive): wire guarded presence-guarantee fallback for headless membership conflicts#3630
Sinity merged 1 commit into
masterfrom
feature/fix/wire-membership-presence-guarantee-fallback

Conversation

@Sinity

@Sinity Sinity commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

Wires _maximal_evidence_fallback into classify_membership_revisions for genuine, irreducible membership conflicts, guarded so it only fires when a logical source has never had ANY accepted head. This is polylogue-lb39z item 5(b)/(c), the last remaining item of the "Raw-authority redesign Phase 1: drain the fake quarantine" bead.

Problem

classify_membership_revisions quarantined every genuine, irreducible membership conflict with accepted_raw_ids == (), even when the logical source had never had any head materialized at all — a document could vanish from the archive permanently just because two revisions couldn't be ordered by containment. _maximal_evidence_fallback (deterministic maximal-evidence pick, order-independent, unit-tested) existed for exactly this but was deliberately never called: wiring it in unconditionally would let a later membership pass silently retire an already-accepted head in favor of an unrelated raw, tripping a real, carefully-designed apply_raw_membership_classification write-back invariant (polylogue-miwv, PR #3211).

Solution

classify_membership_revisions gains an existing_accepted_raw_id: str | None = None keyword. The fallback applies ONLY when existing_accepted_raw_id is None — no head exists yet for the cohort under ANY authority (byte-governed or membership-governed). Both production call sites now pass the cohort's current accepted head raw_id through:

  • polylogue/sources/live/batch.py (incremental live ingest) — already computed accepted_head_raw_id unconditionally at the call site, just wired the pass-through.
  • polylogue/sources/revision_backfill.py (offline rebuild) — head_raw_id was already fetched unconditionally but only conditionally absorbed into the candidate cohort when quarantined; the pass-through to the classifier is now unconditional so the guard also knows about a chain-governed (non-quarantined) existing head it deliberately never absorbs into the comparison.

Non-obvious decision, found during verification, not in the original plan: the guard is deliberately narrower than "only refuse when the fallback would pick a different raw_id than the existing head." A "pure re-affirmation" (fallback pick == existing head's raw_id) is NOT actually safe: apply_raw_membership_classification re-accepting that exact raw_id through membership governance still overwrites the head's own accepted_frontier_kind/generation metadata (e.g. downgrading a byte-governed head to "semantic", generation 17 → 2), even though the pointed-to raw_id never changed. This was caught by a real integration-test regression (test_live_multi_session_divergence_reopens_raw_authority) during development, not a theoretical concern, so the guard refuses ANY existing head, not just a mismatched one.

Updated ~15 existing unit/integration test assertions whose fixtures were genuinely-headless conflicts (no prior accepted head) to reflect the new presence-guarantee resolution instead of asserting quarantine, with inline comments computing the expected fallback pick. Added three explicit guard tests covering: no prior head (fallback applies), prior head at a different raw_id (refuses), and prior head at the SAME raw_id as the fallback's own pick (still refuses, per the finding above).

Verification

devtools test tests/unit/archive/test_session_revision_membership.py \
  tests/unit/sources/test_revision_backfill.py \
  tests/unit/sources/test_live_batch_support.py \
  tests/unit/storage/test_revision_replay.py \
  tests/unit/pipeline/test_ingest_batch.py \
  tests/unit/sources/test_parsers_drive.py \
  tests/unit/sources/test_live_watcher.py

4 failed, 393 passed — the 4 failures (test_full_ingest_writes_archive_with_route_observability, test_full_ingest_skips_durably_excised_content_without_aborting_batch, test_append_multi_session_payload_is_rejected_before_index_write, test_backfill_content_cache_across_pages_reduces_parses_and_matches_uncached_archive) reproduce identically with this change reverted (git stash), confirmed pre-existing/unrelated.

devtools verify --quick

"exit_code": 0 (format/lint/mypy/render all/layering/schema-versioning/raw-authority-frontier-executability all ok).

Out of scope / deferred

  • polylogue-lb39z item 5(a) (attachment-id-stability normalizer) was already fully closed per a prior session on this bead; unchanged here.
  • Items 3/4 of the same bead (append-chain backfill, frontier-executability lint) were completed in prior sessions; unchanged here.
  • No live archive mutation performed or attempted — this is a code-only classifier change verified against synthetic fixtures, same discipline as the rest of this bead's prior sessions.
  • polylogue-6753s (byte-duplicate quarantine supersession, ~4,305 quarantined heads byte-identical to already-indexed raws) is a distinct code path (raws with no logical_source_key at all) and is not touched by this PR.

Ref polylogue-lb39z

…membership conflicts

Problem: classify_membership_revisions quarantined every genuine,
irreducible membership conflict with nothing accepted, even when the
logical source had never had ANY head materialized before -- a document
could vanish from the archive permanently just because two revisions
couldn't be ordered by containment. _maximal_evidence_fallback existed,
was unit-tested, but was deliberately never called (polylogue-lb39z item
5): wiring it in unconditionally would let a later membership pass
silently retire an already-accepted head in favor of an unrelated raw
(polylogue-miwv, PR #3211's write-back invariant), proven by two real
integration-test regressions during earlier work on this bead.

Solution: classify_membership_revisions gains an existing_accepted_raw_id
keyword (default None). The fallback applies ONLY when no head exists yet
for the cohort under ANY authority. This guard is narrower than "only
refuse a different raw_id": a second regression surfaced during
development showed that re-accepting the SAME raw_id through membership
governance still overwrites a byte-governed head's own
accepted_frontier_kind/generation metadata (byte -> semantic), a real
authority downgrade despite the pointed-to raw_id never changing
(test_live_multi_session_divergence_reopens_raw_authority). Both
production call sites (polylogue/sources/live/batch.py's incremental
ingest, polylogue/sources/revision_backfill.py's offline rebuild) now
pass the cohort's current accepted head raw_id (already computed at both
sites) through to the classifier.

Updated ~15 existing unit/integration tests whose fixtures were genuinely
headless conflicts (no prior accepted head) to reflect the new
presence-guarantee resolution instead of asserting quarantine; added
explicit guard tests for the three cases (no prior head applies fallback,
prior head at a different raw_id refuses, prior head at the SAME raw_id
as the fallback's own pick still refuses).

Deferred: item 5(a) of polylogue-lb39z (attachment-id-stability
normalizer) was already fully closed per prior session notes, confirmed
unchanged. Item 4 (frontier-executability lint) and item 3 (append-chain
backfill) were completed in prior sessions on this bead. This PR is the
remainder of item 5(b)/(c).

Verification:
- devtools test tests/unit/archive/test_session_revision_membership.py
  tests/unit/sources/test_revision_backfill.py
  tests/unit/sources/test_live_batch_support.py
  tests/unit/storage/test_revision_replay.py
  tests/unit/pipeline/test_ingest_batch.py
  tests/unit/sources/test_parsers_drive.py
  tests/unit/sources/test_live_watcher.py
  -> "4 failed, 393 passed" -- the 4 failures reproduce identically with
  this change reverted (git stash), confirmed pre-existing/unrelated
  (test_full_ingest_writes_archive_with_route_observability,
  test_full_ingest_skips_durably_excised_content_without_aborting_batch,
  test_append_multi_session_payload_is_rejected_before_index_write,
  test_backfill_content_cache_across_pages_reduces_parses_and_matches_uncached_archive).
- devtools verify --quick -> exit_code 0 (format/lint/mypy/render all/
  layering/schema-versioning/raw-authority-frontier-executability all ok).

Ref polylogue-lb39z
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@Sinity, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 45 seconds

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d3e13335-cc61-49c8-bd82-69a753a69313

📥 Commits

Reviewing files that changed from the base of the PR and between 2643c77 and 2851744.

📒 Files selected for processing (7)
  • polylogue/archive/session_revision_membership.py
  • polylogue/sources/live/batch.py
  • polylogue/sources/revision_backfill.py
  • tests/unit/archive/test_session_revision_membership.py
  • tests/unit/sources/test_live_batch_support.py
  • tests/unit/sources/test_revision_backfill.py
  • tests/unit/storage/test_revision_replay.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Sinity
Sinity merged commit 15c2e54 into master Aug 3, 2026
3 checks passed
@Sinity
Sinity deleted the feature/fix/wire-membership-presence-guarantee-fallback branch August 3, 2026 10:08
Sinity added a commit that referenced this pull request Aug 3, 2026
test_live_batch_support.py: normal.jsonl fixture used bare '{}' which
polylogue-lb39z's guarded presence-guarantee fallback (#3630) now
correctly refuses on real re-parse -- give it a real minimal session.
test_live_watcher.py: os.cpu_count moved from ingest_batch._core to
process_pool.py during today's refactors; 2 monkeypatch targets updated.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant