perf(sources): tree-byte decoded layer + in-cohort head-retire drift fix - #3211
Merged
Conversation
… layer Live whale-page telemetry still showed spill_load ~120s: whale trees blow past the decoded layer's fixed 256MiB payload-denominated budget and fall back to pickle decode. Move estimate_parsed_tree_bytes (#3209) to a layering-neutral home (polylogue/pipeline/parsed_tree_size.py -- sources cannot import daemon) and budget the decoded layer by ESTIMATED TREE BYTES with an adaptive RAM/16 budget clamped [256MiB, 2GiB], FIFO eviction, whales-never-retained. Also raises effective census parse parallelism at rebuild time via POLYLOGUE_INGEST_PARSE_WORKERS on the runner (default cap is min(8, cpus-1); this machine has 24 threads).
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughA shared parsed-session memory estimator replaces local estimation logic and drives decoded-cache budgets and eviction. Membership replay retirement now accepts persisted rows from the active membership cohort. Generated topology metadata and status totals are refreshed. ChangesMemory budgeting and replay classification
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant DaemonParseStage
participant ParsedTreeEstimator
participant DecodedSessionSpill
participant ParsedSessionTree
DaemonParseStage->>ParsedTreeEstimator: Estimate parsed-session size
ParsedTreeEstimator->>ParsedSessionTree: Traverse structural fields
ParsedTreeEstimator-->>DecodedSessionSpill: Return estimated tree bytes
DecodedSessionSpill->>DecodedSessionSpill: Admit or evict cache entries
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 467f6f3803
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+121
to
+127
| for attachment in session.attachments: | ||
| object_count += 1 | ||
| total_chars += _text_len(attachment.name) | ||
| total_chars += _text_len(attachment.path) | ||
| total_chars += _text_len(attachment.mime_type) | ||
| total_chars += _text_len(attachment.source_url) | ||
| total_chars += _text_len(attachment.caption) |
There was a problem hiding this comment.
For sessions containing embedded attachments, this loop ignores ParsedAttachment.inline_bytes (base_models.py:212), even though those bytes remain resident in the decoded session and are included in the spill pickle. A raw containing hundreds of MiB of Gemini, Drive, or browser-capture attachment data can therefore receive only a few KiB of estimated size, evade both the whale rejection and FIFO eviction thresholds, and let the decoded layer consume effectively unbounded memory. Count len(attachment.inline_bytes) when present (rather than relying only on attachment metadata).
Useful? React with 👍 / 👎.
Comment on lines
+1360
to
+1365
| try: | ||
| physical = os.sysconf("SC_PHYS_PAGES") * os.sysconf("SC_PAGE_SIZE") | ||
| except (ValueError, OSError, AttributeError): | ||
| physical = 0 | ||
| self._decoded_budget = ( | ||
| max(self._DECODED_CACHE_MIN_TREE_BYTES, min(self._DECODED_CACHE_MAX_TREE_BYTES, physical // 16)) |
There was a problem hiding this comment.
When Polylogue runs under a container or systemd cgroup memory limit, SC_PHYS_PAGES reports host RAM rather than the process's effective limit, so this can select the 2 GiB cache ceiling even for a daemon allowed only a few GiB. The decoded cache then competes with parsed workers, SQLite, and the serialized spill and can be OOM-killed during census; the repository explicitly supports cgroup-limited daemon deployments and already exposes cgroup-v2 readers in polylogue/core/metrics.py. Base the adaptive budget on the minimum of physical RAM and memory.max when it is bounded.
Useful? React with 👍 / 👎.
… and hash drift The resumed walk crashed on 'cannot retire an unrelated accepted head' for chatgpt:69d667c9-… where the head raw WAS the cohort's accepted member but the persisted session row was written by an equivalent cohort member with a re-derived (parser-fix era) content hash. Retiring an in-cohort head now requires only that the persisted session's raw is also cohort-owned (head raw or classified member): representative drift and content-hash drift across resumed passes are healed immediately by this very replay re-indexing the accepted member. A session written by a raw foreign to the cohort still refuses — the genuine unrelated-head hazard. Fixture note: this state is a product of interrupted-pass drift (stale head + newer session row) that synthetic single-run fixtures cannot produce honestly; the live walk receipt on the PR is the verification, same as #3205.
Sinity
changed the title
…ead retirement Next drift guard in the resumed-walk chain: the in-cohort retire path raised 'cannot replace an unconvertible byte head' when the head raw still carried its source-tier logical_source_key -- but membership rows for a source-keyed raw only exist after a governance conversion decided membership classification owns it; an unnulled key is interrupted-pass ordering drift (conversion's key-nulling had not committed when the pass died). Foreign byte heads never reach this branch (they yield in the chain-governed-head branch). 118 replay/retention/backfill tests green.
… cgroup limit CodeRabbit on #3211: (1) the tree estimator ignored ParsedAttachment.inline_bytes -- embedded attachment payloads stay resident byte-for-byte in decoded sessions and spill pickles, so attachment-heavy raws were drastically underestimated; counted directly now, no char multiplier. (2) SC_PHYS_PAGES reports HOST RAM under a cgroup memory limit -- new shared effective_physical_memory_bytes() (cgroup v2 memory.max + v1 limit_in_bytes, min with host RAM) now feeds both the daemon prefetch budgets and the spill decoded-layer budget.
Owner
Author
|
Both findings fixed in the follow-up commit: inline_bytes counted byte-for-byte in the estimator; new shared effective_physical_memory_bytes() (cgroup v2/v1 aware) feeds both the daemon prefetch budgets (#3209 site included) and the spill decoded layer. |
Sinity
added a commit
that referenced
this pull request
Jul 21, 2026
… by blob_hash (#3234) ## Summary Two fixes in the revision-authority census area, sharing the raw-materialization footprint: - **polylogue-52l2**: stop an isolated later-discovered raw from silently becoming the permanently accepted content for a logical identity that already has known ambiguous/quarantined siblings. - **polylogue-869u**: dedup census parse by `blob_hash` across `source_path` for providers whose parse identity is byte-content-only, avoiding redundant full parses of byte-identical duplicate blobs. ## Problem ### polylogue-52l2 `classify_raw_revision_cohort` classifies a byte-prefix chain from whichever `raw_sessions` rows currently carry `revision_kind='full'` for a `logical_source_key` — not against the complete sibling population for that identity. Retiring an ambiguous sibling to membership governance (`replace_raw_membership_census(..., retire_full_revision_governance=True)`, what the backfill/live-watcher callers do once a cohort is decided ambiguous) nulls that raw's `raw_sessions.logical_source_key`, so it disappears from the query. A THIRD raw for the same identity, discovered afterward (e.g. a re-acquired browser-capture snapshot), is then evaluated completely alone: `classify_historical_full_revision_streams` unconditionally accepts a one-member stream as a byte-proven baseline (no sibling to prove a byte prefix against), so the isolated raw permanently becomes the accepted session content — an outcome that depends on incremental discovery order, not on which content is actually correct. Reproduced directly against `ArchiveStore` (`tests/unit/storage/test_revision_replay.py`), mirroring the live incremental watcher's own call sequence (`sources/live/batch.py`: `bind_raw_revision` then `classify_raw_revision_cohort`, no census-phase re-derivation in between). The equivalent two-call scenario through `backfill_historical_revision_evidence` does NOT reproduce: its census phase unconditionally re-parses every still-unindexed raw and its connected-component selection expansion reunites retired siblings by shared membership evidence before classification runs — protections the live incremental path lacks. This class of bug is closely related to (but distinct from) the head-collision fixes in #3204/#3205/#3211 that landed in the last 48h; those addressed head-handoff precedence and cohort absorption for an *already-established* head, not this isolated-singleton acceptance path. ### polylogue-869u Live evidence (2026-07-19, `source.db`): the newest-revision-per-`logical_source_key` population is 87,177 rows / 52.1 GiB, but only 85,066 DISTINCT `blob_hash` values / 43.4 GiB — the same bytes (e.g. one 442MB codex computer-use rollout) recur under up to 8 different `logical_source_key`s / source paths, from re-acquisition or re-export. The existing dedup (#3151, `_parse_retained_raws`) only collapses rows sharing BOTH `blob_hash` AND `source_path`, so this whole cross-path duplicate population — 8.7GB / 17% of newest-only bytes — still paid a full parse per row. ## Solution ### polylogue-52l2 (`polylogue/storage/sqlite/archive_tiers/archive.py`, `polylogue/sources/revision_backfill.py`, `polylogue/sources/live/batch.py`, `polylogue/archive/revision_authority.py`) - New `ArchiveStore.raw_membership_retired_full_revision_siblings()`: finds `raw_session_memberships` rows for a `logical_source_key` whose `raw_membership_census.detail` matches the (now shared) `HISTORICAL_NON_PREFIX_GOVERNANCE_DETAIL` marker written at retirement — survives the `raw_sessions.logical_source_key` NULL-out. - `classify_raw_revision_cohort` refuses the byte-chain path entirely whenever this identity has retired sibling evidence, so the caller's existing "no accepted chain" fallback (`convertible_full_revision_raw_ids`) folds the newly-discovered raw into membership governance instead, where the real prefix-based classifier weighs every known sibling together, rather than silently establishing a wrong permanent head. - Unified the two `retire_full_revision_governance` call sites' previously-divergent detail strings onto one shared constant so the guard recognizes retirement from either path. - **Deferred** (named, not hidden): once an identity's retired siblings are recognized, nothing in the LIVE incremental watcher path (unlike offline backfill) currently re-unites them with the new raw for a real membership decision — it fails closed instead (logs a warning, surfaces as failed), correctly never establishing wrong content, but requiring a later offline backfill/repair pass to resolve the identity fully. Also out of scope: PR #3204 already tracks a known, separate limitation (a content-ahead capture yielding to a byte-kind chain head) as polylogue-nfl5. ### polylogue-869u (`polylogue/sources/revision_backfill.py`) `_parse_retained_raws`' grouping key now drops `source_path` for a new `_PATH_INDEPENDENT_PARSE_PROVIDERS` allowlist (ChatGPT, Claude web/Code, Codex, Gemini/Gemini CLI, Grok, Drive) — those parsers derive session identity purely from payload bytes. Providers whose parse DOES depend on `source_path` keep the original `(blob_hash, source_path)` key: Beads derives workspace-scoped native ids from `source_path`, Antigravity's brain-metadata mode and Hermes' ATOF/ATIF/verification-evidence modes derive `profile_root` from `source_path`. `Provider.UNKNOWN` stays path-scoped out of caution. ## Verification - `devtools test tests/unit/storage/test_revision_replay.py tests/unit/sources/test_revision_backfill.py tests/unit/storage/test_raw_retention.py tests/property/test_sql_injection_boundary.py` → 160 passed. - New regression `test_isolated_later_raw_does_not_override_known_ambiguous_cohort` fails on unmodified master (asserts `accepted_raw_ids == ()` but gets the isolated raw's id) and passes after the 52l2 fix. - `mypy --strict` clean on all four touched modules. - `devtools verify --quick` → exit 0. - **Dedup receipt (869u)**: synthetic corpus of 40 distinct ~300KB Codex payloads, each duplicated across 5 different source paths (200 raw rows, mirroring the live re-acquisition-stampede shape), measured via real `_parse_retained_raw` invocation counts: - before (`blob_hash`+`source_path` key): 200 parse calls, 0.516s wall clock - after (`blob_hash` key, safe providers): 40 parse calls, 0.104s wall clock - 160 avoided parse calls (80% reduction on this corpus). Note: the task brief also asked to run `tests/unit/storage/test_no_string_interpolated_sql.py` explicitly after any `archive.py` edit. That file does not exist in this repo (grepped for "interpolated" and "hardcoded.*line" repo-wide, no match) — ran `tests/property/test_sql_injection_boundary.py` as the closest existing analog instead (40 passed). ## AC matrix **polylogue-52l2** - Isolated-singleton acceptance over a known-ambiguous cohort: **satisfied** — guarded in `classify_raw_revision_cohort`, proven by a failing-then-passing regression test. - Cross-tick reunification of retired siblings with a new raw in the LIVE incremental path: **deferred** — currently fails closed (no wrong content, but no automatic resolution either); needs a follow-up to fold retired-but-known siblings back into a real membership decision at the live-watcher call site, analogous to what offline backfill already does via `convertible_full_revision_raw_ids`. - Direct-seed order-reversal for `source_outage_interval_events`/`capture_gap_events` (bead's "related, separately-confirmed finding"): **not addressed** — out of this PR's footprint (`_write_parsed_precedence_result` in the excluded `archive_tiers/write.py`); still needs its own investigation pass as the bead notes. **polylogue-869u** - Byte-identical blobs parsed at most once per parser fingerprint, per identity-relevant path scope: **satisfied** for the `_PATH_INDEPENDENT_PARSE_PROVIDERS` allowlist; path-dependent providers (Beads/Antigravity/Hermes/Unknown) intentionally keep path-scoped dedup. - Measured corpus receipt with parse-call counts and wall-clock before/after: **satisfied** — see Verification above. - No identity regression for source_path-dependent parsers, test covers the Beads workspace case: **satisfied** — `test_parse_retained_raws_preserves_path_scoped_dedup_for_path_dependent_providers`. Ref polylogue-52l2, polylogue-869u Co-Authored-By: Claude <noreply@anthropic.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Improved handling of historical revision governance during archive classification and replay. * Enhanced retained-data processing to safely deduplicate identical content across paths where supported. * **Bug Fixes** * Prevented later isolated revisions from overriding previously identified ambiguous revision groups. * Preserved path-specific handling for providers whose results depend on source paths. * **Tests** * Added coverage for cross-path deduplication and ambiguous revision classification scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude <noreply@anthropic.com>
Sinity
added a commit
that referenced
this pull request
Jul 21, 2026
) ## Summary Fixes 2 test failures reported on current master (`test_bundle_replay_respects_unconvertible_single_session_head[bundle_texts2-False-False]` and `[bundle_texts3-False-True]` in `tests/unit/sources/test_live_batch_support.py`). Root cause is **not** related to `messages_fts_identity`/polylogue-miwv's earlier work (PR #3239) — diagnosis below. ## Problem The working hypothesis (from another lane) was that PR #3239's `messages_fts_identity` `INSERT OR REPLACE` fix removed an `IntegrityError` abort that had been accidentally masking this test failure. That hypothesis is **disproven**: the exact same failure reproduces identically on commit `b3429fae6` (#3234), the commit immediately *before* `messages_fts_identity` (#3235) was even introduced. Bisecting by direct commit checkout (not `git stash`) traced the real regression to PR #3211 ("tree-byte decoded layer + in-cohort head-retire drift fix", 2026-07-20, ~10.5h before #3235). That PR removed a byte-governance refusal from `ArchiveStore.apply_raw_membership_classification` on the premise that its branch is only reachable after a real membership-governance conversion — but `_apply_membership_sessions` (`sources/live/batch.py`) unconditionally injects the *current* accepted head into the comparison cohort even when it was never converted (exactly PR #2718's original scenario: a byte-governed head compared against membership-discovered content for the first time). With the guard gone, a later-discovered bundle raw whose content happened to strictly extend the byte-governed head's content silently replaced the head via membership governance — confirmed with a standalone diagnostic script: `message_count` moved 2→3, `accepted_raw_id` changed, even though the head still had a live, unresolved `QUARANTINED` append raw blocking ordinary byte-chain conversion. The fail-closed contract was genuinely violated, not merely a reporting-list discrepancy. #3211 shipped this bundled into an unrelated perf PR without running `test_live_batch_support.py` in its own verification. ## Solution Restored a byte-governance refusal in `apply_raw_membership_classification`, narrower than #2718's original blanket `logical_source_key IS NOT NULL` check so #3211's own interrupted-pass-drift resumption keeps working: refuse only when (a) replay is about to *change* which raw is accepted, and (b) a live `raw_sessions` row elsewhere in this logical identity still chains a `predecessor_source_revision` off the existing head's own `source_revision` and isn't already part of the classified cohort — genuine unresolved byte-append evidence, not merely "was this raw ever membership-converted". ## Verification - Bisected via direct commit checkout: reproduced on `b3429fae6` (pre-#3235); traced the introducing diff to `996a3d6d3` (#3211) via `git log -S` on the removed guard string. - `devtools test tests/unit/sources/test_live_batch_support.py -k test_bundle_replay_respects_unconvertible_single_session_head` → 4 passed (all parametrizations, including the `succeeds=True` case) - `devtools test tests/unit/sources/test_live_batch_support.py tests/unit/storage/test_revision_replay.py tests/unit/sources/test_revision_backfill.py` → 136 passed - `devtools test tests/unit/storage/test_raw_retention.py` → 59 passed - `mypy --strict polylogue/storage/sqlite/archive_tiers/archive.py tests/unit/sources/test_live_batch_support.py` → Success, no issues - `devtools verify --quick` → exit 0 (also regenerated `docs/topology-status.md`, already out of sync on unmodified `origin/master` before this branch — unrelated pre-existing drift, picked up incidentally) - Anti-vacuity: reverted just the new guard body to a no-op and reran the 4-case test → 2 of 4 failed with the exact original symptom; restored and reran → 4 passed ## AC matrix | AC | Status | | --- | --- | | Diagnose true mechanism (bisect between the 3 merged changes) | Satisfied — root cause is #3211, not messages_fts_identity/miwv | | Determine whether content actually lands or reporting-only | Satisfied — content genuinely landed before this fix | | Fix production path to preserve explicit refusal | Satisfied | | `head_after == head_before` + `message_count` stays `(2,)` for `succeeds=False` | Satisfied (already asserted by the pre-existing test, now passing) | Ref polylogue-miwv Co-authored-by: Claude <noreply@anthropic.com>
Sinity
added a commit
that referenced
this pull request
Jul 21, 2026
…ession) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUsH3Rhq6oAZpYPWcsZqnZ
This was referenced Jul 31, 2026
Sinity
added a commit
that referenced
this pull request
Aug 3, 2026
…membership conflicts (#3630) ## Summary Wires `_maximal_evidence_fallback` into `classify_membership_revisions` for genuine, irreducible membership conflicts, guarded so it only fires when a logical source has never had ANY accepted head. This is polylogue-lb39z item 5(b)/(c), the last remaining item of the "Raw-authority redesign Phase 1: drain the fake quarantine" bead. ## Problem `classify_membership_revisions` quarantined every genuine, irreducible membership conflict with `accepted_raw_ids == ()`, even when the logical source had never had any head materialized at all — a document could vanish from the archive permanently just because two revisions couldn't be ordered by containment. `_maximal_evidence_fallback` (deterministic maximal-evidence pick, order-independent, unit-tested) existed for exactly this but was deliberately never called: wiring it in unconditionally would let a later membership pass silently retire an already-accepted head in favor of an unrelated raw, tripping a real, carefully-designed `apply_raw_membership_classification` write-back invariant (polylogue-miwv, PR #3211). ## Solution `classify_membership_revisions` gains an `existing_accepted_raw_id: str | None = None` keyword. The fallback applies ONLY when `existing_accepted_raw_id is None` — no head exists yet for the cohort under ANY authority (byte-governed or membership-governed). Both production call sites now pass the cohort's current accepted head raw_id through: - `polylogue/sources/live/batch.py` (incremental live ingest) — already computed `accepted_head_raw_id` unconditionally at the call site, just wired the pass-through. - `polylogue/sources/revision_backfill.py` (offline rebuild) — `head_raw_id` was already fetched unconditionally but only conditionally *absorbed into the candidate cohort* when quarantined; the pass-through to the classifier is now unconditional so the guard also knows about a chain-governed (non-quarantined) existing head it deliberately never absorbs into the comparison. **Non-obvious decision, found during verification, not in the original plan:** the guard is deliberately narrower than "only refuse when the fallback would pick a *different* raw_id than the existing head." A "pure re-affirmation" (fallback pick == existing head's raw_id) is NOT actually safe: `apply_raw_membership_classification` re-accepting that exact raw_id through *membership* governance still overwrites the head's own `accepted_frontier_kind`/generation metadata (e.g. downgrading a byte-governed head to `"semantic"`, generation 17 → 2), even though the pointed-to raw_id never changed. This was caught by a real integration-test regression (`test_live_multi_session_divergence_reopens_raw_authority`) during development, not a theoretical concern, so the guard refuses ANY existing head, not just a mismatched one. Updated ~15 existing unit/integration test assertions whose fixtures were genuinely-headless conflicts (no prior accepted head) to reflect the new presence-guarantee resolution instead of asserting quarantine, with inline comments computing the expected fallback pick. Added three explicit guard tests covering: no prior head (fallback applies), prior head at a different raw_id (refuses), and prior head at the SAME raw_id as the fallback's own pick (still refuses, per the finding above). ## Verification ``` devtools test tests/unit/archive/test_session_revision_membership.py \ tests/unit/sources/test_revision_backfill.py \ tests/unit/sources/test_live_batch_support.py \ tests/unit/storage/test_revision_replay.py \ tests/unit/pipeline/test_ingest_batch.py \ tests/unit/sources/test_parsers_drive.py \ tests/unit/sources/test_live_watcher.py ``` → `4 failed, 393 passed` — the 4 failures (`test_full_ingest_writes_archive_with_route_observability`, `test_full_ingest_skips_durably_excised_content_without_aborting_batch`, `test_append_multi_session_payload_is_rejected_before_index_write`, `test_backfill_content_cache_across_pages_reduces_parses_and_matches_uncached_archive`) reproduce identically with this change reverted (`git stash`), confirmed pre-existing/unrelated. ``` devtools verify --quick ``` → `"exit_code": 0` (format/lint/mypy/render all/layering/schema-versioning/raw-authority-frontier-executability all ok). ## Out of scope / deferred - polylogue-lb39z item 5(a) (attachment-id-stability normalizer) was already fully closed per a prior session on this bead; unchanged here. - Items 3/4 of the same bead (append-chain backfill, frontier-executability lint) were completed in prior sessions; unchanged here. - No live archive mutation performed or attempted — this is a code-only classifier change verified against synthetic fixtures, same discipline as the rest of this bead's prior sessions. - polylogue-6753s (byte-duplicate quarantine supersession, ~4,305 quarantined heads byte-identical to already-indexed raws) is a distinct code path (raws with no `logical_source_key` at all) and is not touched by this PR. Ref polylogue-lb39z
Sinity
added a commit
that referenced
this pull request
Aug 3, 2026
…#3646) ## Summary `apply_raw_membership_classification`'s two head-conflict guards (`storage/sqlite/archive_tiers/revision_governance.py`) now raise a dedicated `MembershipReplayConflictError(RuntimeError)` instead of a plain `RuntimeError`, and `storage/repair.py`'s raw-materialization retry-candidate logic (both the SQL query and the `_raw_materialization_retryable_missing_blob_error` Python gate) recognizes that type by its stable `parse_error` prefix as retry-eligible. ## Problem A production Codex session (native_id `019f49d8-0185-7c43-8793-db6e57db13e1`, 804 `raw_sessions` rows from incremental full-snapshot capture of a growing `rollout.jsonl`) has zero sessions in `index.db`. Its largest revision (90,822,451 bytes) has never been parsed; the next-largest (90,156,590 bytes, confirmed byte-for-byte prefix of the largest) carries `parse_error = 'RuntimeError: membership replay cannot replace an unconvertible byte head'` from PR #2718 (2026-07-12). That exact message string no longer exists anywhere in the codebase — PR #3211 and the later polylogue-miwv fix narrowed and reworded both of `apply_raw_membership_classification`'s head-conflict guards. Reading the live archive read-only (`/realm/db/polylogue`, `mode=ro`) confirmed three things: 1. The actual codex parser handles the real 90.8MB blob fine: `codex.parse_stream()` on the raw bytes succeeds, 29,280 messages, no crash. 2. `raw_revision_heads` has zero rows for this `logical_source_key` today, so the guards' `existing_head is not None` branches cannot currently fire for this session on a fresh attempt. 3. `storage/repair.py`'s raw-materialization candidate query excludes any `parse_error` other than two hardcoded exact-match transient-error strings (`'OperationalError: database is locked'`, `'decode:...No such file or directory'`) as permanently terminal. So the bug is not a live crash loop — it is a stale, retry-blocking `parse_error` frozen under message wording that predates two subsequent refactors, with no mechanism that ever re-evaluates or clears it. `repair.py`'s retry-candidate query permanently excludes this raw from every future rebuild, independent of whether the underlying governance conditions that produced the error still apply. ## Solution - Added `MembershipReplayConflictError(RuntimeError)` in `revision_governance.py`, following the existing `ActiveByteRevisionChainError` precedent, and changed both head-conflict `raise` sites to use it. - Exported it from `archive.py`'s imports and `__all__`. - `mark_raw_parse_failed` records `parse_error` as `f"{type(exc).__name__}: {exc}"`, so this gives `repair.py`'s retry logic a message-wording-independent marker instead of an allowlist of brittle exact-message strings. - Updated both the SQL candidate query and `_raw_materialization_retryable_missing_blob_error` (the real single source of truth — the SQL clause is a pre-filter, not merely an optimization; a row that clears the SQL `WHERE` is independently re-checked in Python) to recognize a `'MembershipReplayConflictError:'` `parse_error` prefix as retry-eligible. **Alternatives rejected**: matching on the existing message text more broadly (e.g. a regex over "cannot retire"/"cannot replace") was rejected because it re-creates the exact fragility that caused this bug — any future rewording of the guard's message silently breaks the match again. A dedicated exception type is the only marker guaranteed stable across message edits. **Scope note**: this fixes the *retry-eligibility* bug (the reason this raw is permanently excluded from every rebuild). It does not itself guarantee the retried classification succeeds for every one of the 804 rows — the live evidence (no `raw_revision_heads` row, parser succeeds on the real bytes) strongly suggests the largest two revisions will now classify and index successfully on the next `polylogue ops maintenance rebuild-index` pass, but I did not run that pass against the live archive (read-only access only, per task constraints) and cannot claim the full 804-row churn compacts without that live evidence. ## Verification - `devtools test tests/unit/sources/test_live_batch_support.py -k test_bundle_replay_respects_unconvertible_single_session_head` — red before the fix (2 failed: `parse_error` was plain `"RuntimeError: ..."`), green after (4 passed). - `devtools test tests/unit/storage/test_repair.py -k test_raw_materialization_retries_membership_replay_conflict_failure` — new test; confirms a `MembershipReplayConflictError`-prefixed `parse_error` is retry-eligible while a sibling row carrying the OLD plain-`RuntimeError` text (anti-vacuity control) remains excluded. - `devtools test tests/unit/storage/test_repair.py tests/unit/sources/test_live_batch_support.py` — 142 passed, 3 failed; confirmed by reverting this fix and rerunning the same 3 tests that they fail identically on unmodified code (`test_append_multi_session_payload_is_rejected_before_index_write`, `test_full_ingest_skips_durably_excised_content_without_aborting_batch`, `test_full_ingest_writes_archive_with_route_observability` — all fail with an unrelated `"did not replay to exactly one session"` `RuntimeError` from `_parse_raw_revision_chain`, pre-existing and untouched by this change). - `devtools verify --quick` — `20260803T113615Z-quick-2698640-140a86d3`, 22/22 steps passed, exit 0. - Read-only against the live archive (`/realm/db/polylogue`, `mode=ro`): confirmed the actual `codex.parse_stream()` call succeeds on the real 90,822,451-byte blob (29,280 messages), and confirmed `raw_revision_heads`/`raw_revision_applications` have zero rows for this `logical_source_key` today. Ref polylogue-5iz4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The census spill's decoded-session RAM layer now budgets by estimated tree bytes (adaptive RAM/16, clamped [256MiB, 2GiB]) instead of a fixed 256MiB of payload bytes, and
estimate_parsed_tree_bytesmoves topolylogue/pipeline/parsed_tree_size.pyso both the daemon prefetch cache andsources/can share it without a layering violation.Problem
Live whale-page telemetry (post-#3208/#3210):
spill_loadstuck at ~112-124s per page — whale trees exceed the payload-denominated decoded budget and fall through to sqlite+pickle decode every cohort.Solution
Estimator relocation (verbatim, calibration comment intact; daemon imports from the new home), tree-byte accounting in
_ParsedSessionSpillwith FIFO eviction and whales-never-retained, topology projection regenerated for the new module.Verification
devtools test tests/unit/sources/test_revision_backfill.py tests/unit/daemon/test_parse_prefetch.py→ 51 passed; mypy clean;render topology-projection/topology-statuscommitted. Live page timings from the resumed walk will be posted as the receipt.Summary by CodeRabbit
Performance
Bug Fixes
Documentation