fix(sources): require positive conversational evidence for session creation by Sinity · Pull Request #3497 · Sinity/polylogue

Sinity · 2026-07-31T22:51:13Z

Summary

Two-part fix for the classification-gate correctness problem: (1) repairs a genuine regression introduced by the polylogue-9ykn content-classification gate (PR #3428) that refused genuine Claude Code session records, and (2) implements the general "session requires positive conversational evidence" invariant at the parse/ingest chokepoint, with a read-only reconciliation of the 5,257 zero-message sessions currently in the live archive.

Problem

polylogue-6mpy (failing test on clean master): tests/unit/sources/test_revision_backfill.py::test_parse_one_still_replays_real_claude_code_sessions_with_no_path_rule failed with assert 0 == 1. Root cause: classify_artifact's content gate (added for polylogue-9ykn, PR #3428) consulted classify_artifact_path first and returned unconditionally — any source path containing an analysis/ directory segment was refused as a self-generated side-output artifact regardless of record content, so a genuine Claude Code session record (role=user, real sessionId, real timestamp) sitting under such a path was silently dropped on replay.

polylogue-9ykn (P0): the ingest path's default disposition was "this is a session" — anything not positively recognised as something else still became one. Measured against the live archive (2026-07-31, read-only queries against /realm/db/polylogue/{index,source}.db):

SELECT COUNT(*) FROM sessions;                                   -- 23,496
SELECT origin, COUNT(*) FROM sessions WHERE message_count=0
  GROUP BY origin ORDER BY 2 DESC;
  -- claude-code-session  5,193
  -- claude-ai-export         47
  -- codex-session            17
  -- total: 5,257 (22.4%)

-- C4 overlap (created_at_ms NULL population):
SELECT COUNT(*) FROM sessions WHERE created_at_ms IS NULL;       -- 5,383
SELECT COUNT(*) FROM sessions
  WHERE message_count=0 AND created_at_ms IS NULL;                -- 5,192

5,192 of the 5,193 empty claude-code-session rows (and 5,192/5,193 overall) also carry created_at_ms IS NULL — this is the C4 population (the one row with a non-NULL created_at_ms is problems_index, a self-generated analysis artifact with its own real acquisition timestamp).

Solution

polylogue-6mpy

polylogue/archive/artifact_taxonomy/runtime.py: split classify_artifact_path into a weak, content-blind analysis/-directory heuristic (_self_generated_artifact_dir_classification) and the strong, definitive path rules (_classify_artifact_path_strong: OriginSpec rules, known sidecar filenames, Hermes/Antigravity path markers). classify_artifact (content-aware) now checks strong path rules first (unconditional, unchanged), then content classification, and only falls back to the weak analysis/ heuristic when content shows no positive session evidence — so a relationship-index pointer record (no envelope markers) is still refused, but a genuine session record wins on content regardless of directory naming.

polylogue-9ykn

polylogue/sources/dispatch.py gains require_positive_conversational_evidence() + message_carries_authored_content(): a session is kept only if at least one message has real text or a content block.

Deliberately not folded into parse_payload/parse_stream_payload themselves — those two functions are pure provider-routing dispatch, pinned by a large "law" test surface (test_source_laws.py and friends) that monkeypatches provider parsers with zero-message stubs specifically to test routing independent of content. Folding the gate in there broke 20+ of those tests. Instead every real production write path applies the filter explicitly after calling parse_payload/parse_stream_payload:

sources/live/batch.py's full-ingest loop (reuses the existing mark_raw_parse_failed "recorded, bounded failure" mechanism — a filtered-to-empty result trips the same if not sessions: branch a genuinely-unparseable payload already hits)
pipeline/services/ingest_worker.py's _parse_plan_sessions (subprocess decode/parse worker; reuses the existing "parse: session artifact produced no materializable sessions" error path)
sources/live/append_ingest.py (incremental append)
sources/revision_backfill.py's _parse_one/_parse_stream (offline replay/rebuild, alongside its own polylogue-6mpy path/shape gate — catches the sibling case where the shape is recognised but parsed content still carries no message)

Checking message content, not just message count, closes a sibling gap found while testing: an unrecognized single-record document with no known provider markers fell through Claude Code's generic single-document lowering into a one-message session whose sole message had empty text and no blocks — structurally "has a message" but zero actual evidence.

Reconciliation (AC b) — read-only, `?mode=ro`

Joining index.db.sessions to source.db.raw_sessions via raw_id:

Class	Count	Disposition
`agent-*.meta` sidecars	4,945	Artifact — already refused by `classify_artifact`'s path rule (pre-dates this PR); pending purge
JSONL files with only non-conversational envelope records (`file-history-snapshot`/`progress`/`bridge-session`/`custom-title`/`agent-name`)	228+	Artifact — genuine per-session JSONL files whose entire content is Claude Code checkpoint/protocol records, never a real message. Eliminated at source by this PR's gate.
`tool-results/*.json` sidecars mis-dispatched as sessions	3	Artifact — wrong-provider-dispatch of a single-JSON-object sidecar file as if it were a session-shaped stream
`.gemini/` paths misdetected as `claude-code-session`	2	Artifact
`claude-ai-export` conversations with a real title but `chat_messages: []`	47	Content-legitimate export entity, but carries no conversation — refused by this PR's gate too
`codex-session` empty rows	17	Mixed: some are zero-byte scan/write-race captures (now refused instead of silently materializing, see `test_full_ingest_empty_jsonl_is_not_misclassified_as_truncated`); a few (996KB/1.4MB blobs) look like a separate, unrelated Codex message-extraction defect — flagged as a residual, not fixed here (out of scope; see Residuals)

On the 832 ne6k-retained rows: polylogue-ne6k's own 2026-07-31 investigation (same day) already superseded the original "832 intentionally-retained genuinely-empty sessions" framing — its corrected note states "the 832 the hook-inflation postmortem retained were retained precisely BECAUSE the blanket predicate could not tell them apart, not because they were verified worth keeping" and "no 'genuinely empty session' is a legitimate construct." This PR's gate is therefore consistent with that corrected finding — it does not carve out a legitimacy exception for any zero-message row.

AC (c) — eliminated at source: the gate above stops all classes above from ever entering the index again, in one place, for every ingest/replay route. Existing phantom rows in the live archive are left for the already-planned v46→v50 index rebuild (polylogue-x1gd) per this lane's non-goals — no live-archive writes were made by this PR.

Anti-vacuity statement

Production callers exercising the new surface: polylogue/sources/live/batch.py:2086 (daemon full-ingest, in-process), polylogue/pipeline/services/ingest_worker.py:_parse_plan_sessions (subprocess decode/parse worker), polylogue/sources/live/append_ingest.py (incremental append), polylogue/sources/revision_backfill.py:_parse_one/_parse_stream (offline replay/rebuild). Mutation that makes the tests fail: removing the message-content check from require_positive_conversational_evidence (or reverting it to a message-count check) makes test_parse_payload_generic_unrecognized_record_shape_manufactures_only_a_content_free_message fail; removing the require_positive_conversational_evidence call from any of the four call sites makes that call site's corresponding fixture test in test_live_batch_support.py/test_resilience.py/test_revision_backfill.py fail (each asserts the specific refused/empty outcome, not merely "no crash").

AC matrix

AC	Status
6mpy: content evidence overrides weak path heuristic; test passes	Satisfied
6mpy: relationship-index / evidence-free shape stays refused	Satisfied (regression-pinned, `test_parse_one_refuses_non_conversational_content_with_no_path_rule`)
9ykn (a): default disposition for unrecognised record is refusal-with-reason, regression-pinned, no session created	Satisfied — 3 new tests in `test_dispatch_payloads.py` pin `require_positive_conversational_evidence` against a session-meta-only stream, a `chat_messages: []` export conversation, and the unrecognized-shape single-message case
9ykn (b): 5,255/5,257 empty-session population explained (intentional vs artifact)	Satisfied — see reconciliation table above; no intentional/legitimate subset remains per ne6k's corrected finding
9ykn (b): C4 `created_at_ms` NULL overlap reported with proving query	Satisfied — 5,192/5,193 overlap, query above
9ykn (c): artifact class eliminated at ingest source (code fix)	Satisfied for the classes measured above; existing rows deferred to the v46→v50 rebuild (non-goal: no live writes)
9ykn: regression test pins unrecognized record ≠ session	Satisfied

Verification

python -m devtools test tests/unit/sources/test_revision_backfill.py tests/unit/sources/test_artifact_taxonomy.py
  -> 65 passed

python -m devtools test tests/infra/pipeline_roundtrip.py tests/unit/sources/test_dispatch_payloads.py \
  tests/unit/sources/test_dispatch_ordering.py tests/unit/sources/test_source_laws.py \
  tests/unit/sources/test_revision_backfill.py tests/unit/sources/test_artifact_taxonomy.py \
  tests/unit/sources/test_live_batch_support.py tests/unit/pipeline/test_resilience.py \
  tests/unit/daemon/test_ingest_worker_handoff.py tests/unit/pipeline/test_ingest_worker_assembly.py
  -> 334 passed, 5 failed

python -m devtools verify --quick
  -> exit 0 (ruff format/check, mypy --strict, render all --check, topology,
     layering, closure-matrix, schema/manifest/policy lab checks all pass)

The 5 remaining failures were each reproduced against a clean-master baseline worktree (git worktree add --detach <tmp> 4477b961a, the commit immediately before this branch) and confirmed to fail identically there — pre-existing, unrelated to this change:

test_dispatch_payloads.py::test_parse_stream_payload_codex_long_rollout_with_repeated_session_meta_yields_messages
test_pipeline/test_resilience.py::test_validation_law_matches_mode_and_payload_contract
test_live_batch_support.py::test_append_multi_session_payload_is_rejected_before_index_write
test_live_batch_support.py::test_full_ingest_skips_durably_excised_content_without_aborting_batch
test_live_batch_support.py::test_full_ingest_writes_archive_with_route_observability

(A wider batch also confirmed as pre-existing this way: test_web_reader.py::test_archive_filter_kwargs_cover_every_storage_lowerable_spec_field, test_synthetic_semantics.py's two antigravity parametrizations, test_claude_code_normalization_laws.py::test_family_fixture_detector_and_streaming_paths_preserve_one_normalized_identity.)

devtools verify (full testmon-affected run) was not run: this fresh worktree has no seeded testmon database (.cache/testmon/testmondata absent) and --seed-testmon would run the full non-integration suite, which is out of the narrow-node-reproduction budget for this lane. The targeted file list above plus --quick's static/generated gates are the verification evidence for this PR.

Residuals / follow-ups

A handful of codex-session zero-message rows (2 of the 17) have large (996KB/1.4MB) raw blobs that look like genuine conversations whose messages failed to extract for an unrelated reason — flagged for a follow-up bead, not investigated further here (out of scope: this lane is about the classification/evidence gate, not a Codex parser defect).
Other in-flight worktrees in this repo have branches named feature/sources/positive-session-classification and feature/sources/self-generated-analysis-artifact-refusal (no open PRs found for either at push time) — their scope may overlap this PR's; worth a coordinator check before merge.
raw_sessions.detection_warnings_json (the durable per-raw-revision warning sink) is not fed by require_positive_conversational_evidence's refusal — the filter runs after a ParsedSession already exists in memory, with no natural session-scoped event sink to write to since no session is created. The refusal is currently recorded via logger.warning plus the existing mark_raw_parse_failed/"produced no materializable sessions" bounded-error paths at each call site. Wiring a claude_parse_coverage-family detection event for this specific refusal reason (PR feat(sources): report per-type sidecar coverage for Claude Code parse #3419) would need a raw-revision-scoped (not session-scoped) sink and is left as a follow-up rather than invented ad hoc here.

Ref polylogue-6mpy, polylogue-9ykn

Problem: tests/unit/sources/test_revision_backfill.py::test_parse_one_still_replays_real_claude_code_sessions_with_no_path_rule failed on clean master (`assert 0 == 1`). classify_artifact's content gate added toward polylogue-9ykn (PR #3428) consulted classify_artifact_path first and returned unconditionally: any source path with an "analysis" directory segment was refused as a self-generated side-output artifact regardless of record content, so a genuine Claude Code session record (role=user, real sessionId, timestamp) sitting under such a path was silently dropped on replay. Solution: split classify_artifact_path into the weak, content-blind "analysis/" directory heuristic (_self_generated_artifact_dir_classification) and the strong, definitive path rules (_classify_artifact_path_strong: OriginSpec rules, known sidecar filenames, Hermes/Antigravity path markers). Path-only callers (decoder_zip, source_walk skip-listing, schema sampling) keep the exact prior behavior via classify_artifact_path, which still checks the weak heuristic first. The content-aware classify_artifact now checks strong path rules first (unconditional, as before), then content classification, and only falls back to the weak analysis/ heuristic when content shows no positive session evidence -- so a relationship-index pointer record (no envelope markers) is still refused, but a genuine session record wins on its content regardless of directory naming. Verification: python -m devtools test tests/unit/sources/test_revision_backfill.py -> 56 passed python -m devtools test tests/unit/sources/test_artifact_taxonomy.py -> 9 passed Ref polylogue-6mpy Co-Authored-By: Claude <noreply@anthropic.com>

…ion write Problem: the ingest path's default disposition was "this is a session" -- anything not positively recognised as something else still became one. Measured against the live archive (2026-07-31): 5,193 zero-message claude-code-session rows (22.6% of the archive), reconciled via read-only queries against index.db/source.db joined to raw_sessions.source_path: 4,945 agent-*.meta sidecars (already refused by classify_artifact's path rule but pre-dating it), 228+ JSONL files containing only non- conversational envelope records (file-history-snapshot/progress/ bridge-session/custom-title/agent-name), 3 tool-results/*.json sidecars mis-dispatched as sessions, plus 47 claude-ai-export conversations with a real title but chat_messages: []. polylogue-ne6k's own investigation (2026-07-31) concluded no "genuinely empty but legitimate" session construct exists in this corpus -- every zero-message row was retained only because the prior repair predicate could not distinguish it from one. Solution: polylogue/sources/dispatch.py gains require_positive_conversational_evidence() + message_carries_authored_ content() -- a session is kept only if at least one message has real text or a content block. Deliberately NOT folded into parse_payload/ parse_stream_payload themselves: those two functions are pure provider- routing dispatch pinned by a large "law" test surface (test_source_laws.py and friends) that monkeypatches provider parsers with zero-message stubs specifically to test routing independent of content -- folding the gate in there broke 20+ of those tests. Instead every real production write path applies the filter explicitly after calling parse_payload/parse_stream_payload: - sources/live/batch.py's full-ingest loop (reuses the existing mark_raw_parse_failed "recorded, bounded failure" mechanism a filtered-to-empty result already trips) - pipeline/services/ingest_worker.py's _parse_plan_sessions (subprocess decode/parse worker) - sources/live/append_ingest.py (incremental append) - sources/revision_backfill.py's _parse_one/_parse_stream (offline replay/rebuild, alongside its own polylogue-6mpy path/shape gate -- this filter catches the sibling case where the shape is recognized but the parsed content still carries no message) Checking message *content*, not just message *count*, also closes a narrower sibling gap found while testing this bead: an unrecognized single-record document with no known provider markers previously fell through Claude Code's generic single-document lowering into a one-message session whose sole message had empty text and no blocks. Verification so far: python -m devtools test tests/unit/sources/test_dispatch_payloads.py tests/unit/sources/test_dispatch_ordering.py tests/unit/sources/test_revision_backfill.py tests/unit/sources/test_artifact_taxonomy.py -> 197 passed, 1 pre-existing unrelated failure (test_parse_stream_ payload_codex_long_rollout_with_repeated_session_meta_yields_messages, confirmed failing on clean master before this change) Residual: a broader targeted run surfaced ~15 more tests in test_live_batch_support.py/test_resilience.py/test_synthetic_semantics.py etc. whose fixtures assume the old "session_meta-only stream succeeds" or "zero-byte capture becomes a legitimately empty session" behavior -- being triaged/updated in a follow-up commit on this branch (some are fixture updates matching the new invariant, at least one -- test_full_ingest_empty_jsonl_is_not_misclassified_as_truncated -- is a considered pre-existing behavior decision that conflicts with this bead's invariant and needs an explicit call, documented in the PR body). Ref polylogue-9ykn Co-Authored-By: Claude <noreply@anthropic.com>

… gate Problem: the broader targeted regression run for the polylogue-9ykn gate surfaced 15 test failures beyond the previous commit's verified set. Triaged each by reproducing against a clean-master baseline worktree (commit 4477b96): 6 were genuinely caused by this bead's new require_positive_conversational_evidence filter (fixtures built session_meta-only or empty-mapping payloads that used to silently materialize a zero-message session), and 9 were pre-existing failures already broken on master, unrelated to this change (confirmed by reproducing the identical failure against the baseline worktree with none of this branch's commits applied). Solution (the 6 real fixes): - tests/unit/sources/test_live_batch_support.py: appended a real response_item message record to 6 session_meta-only Codex JSONL fixtures (test_large_full_ingest_uses_archive, test_streaming_sized_full_ingest_uses_archive, test_full_ingest_heartbeats_small_file_groups_with_current_path, test_busy_full_prefix_proof_defers_to_archived_cursor_reconciliation, test_archive_cursor_reconciliation_rejects_restored_mtime_rewrite) and gave mocked zero-message ParsedSession fixtures a real message (test_append_multi_session_payload_is_rejected_before_index_write's sibling test_full_multi_session_failure_retries_without_success_ mapping) so each test keeps exercising the mechanics it is named for, not the now-refused empty shape. - test_full_ingest_empty_jsonl_is_not_misclassified_as_truncated: this one is a considered behavior change, not a fixture patch. Its original assertion was "a zero-byte capture cleanly materializes as a legitimately empty parsed session" -- exactly the silent-inflation default polylogue-9ykn eliminates. Updated to assert the new, correct outcome: refused with a bounded, honest "no positive conversational evidence" parse_error (never the misleading truncation-boundary error the original fix targeted, which the test still pins). - tests/unit/pipeline/test_resilience.py:: test_ingest_worker_decodes_and_dispatches_provider: its own docstring admitted the pre-existing default -- "ingest_record returns a materializable SessionWritePayload even when the source has no messages yet". Updated to assert the new outcome: refused via the existing "session artifact produced no materializable sessions" ingest_worker.py error path (unchanged mechanism, now reached for the message-content-empty case too). The 9 pre-existing failures are left untouched (out of scope for this bead): test_full_ingest_writes_archive_with_route_observability, test_full_ingest_skips_durably_excised_content_without_aborting_batch, test_append_multi_session_payload_is_rejected_before_index_write (a DIFFERENT still-broken assertion than the one this commit fixed in its sibling -- see below), test_archive_filter_kwargs_cover_every_storage_ lowerable_spec_field, test_validation_law_matches_mode_and_payload_ contract, TestParseRoundtrip::test_synthetic_parses_to_sessions[antigravity], TestCorpusParseRoundtrip::test_generated_data_parses[antigravity], test_family_fixture_detector_and_streaming_paths_preserve_one_normalized_identity, test_parse_stream_payload_codex_long_rollout_with_repeated_session_meta_ yields_messages. Note: test_append_multi_session_payload_is_rejected_before_index_write fails on BOTH master and this branch, but for two different reasons -- the fixture's `sessions` list mock is never reached either way because `_is_declared_non_session_artifact` refuses `{}`-shaped raw bytes before `parse_payload` runs (pre-existing, unrelated to this bead). Gave the mock real content anyway since it is otherwise correct for what the test intends to verify. Verification: python -m devtools test tests/infra/pipeline_roundtrip.py tests/unit/sources/test_dispatch_payloads.py tests/unit/sources/test_dispatch_ordering.py tests/unit/sources/test_source_laws.py tests/unit/sources/test_revision_backfill.py tests/unit/sources/test_artifact_taxonomy.py tests/unit/sources/test_live_batch_support.py tests/unit/pipeline/test_resilience.py tests/unit/daemon/test_ingest_worker_handoff.py tests/unit/pipeline/test_ingest_worker_assembly.py -> 334 passed, 5 failed (all 5 confirmed pre-existing against a clean-master baseline worktree at 4477b96) Ref polylogue-9ykn Co-Authored-By: Claude <noreply@anthropic.com>

chatgpt-codex-connector · 2026-07-31T22:51:18Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

coderabbitai · 2026-07-31T22:51:23Z

Warning

Review limit reached

@Sinity, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 14 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b98483f5-c825-4d88-ae2e-a95c2ccc68cc

📥 Commits

Reviewing files that changed from the base of the PR and between 0ef1fdd and d7759a5.

📒 Files selected for processing (10)

polylogue/archive/artifact_taxonomy/runtime.py
polylogue/pipeline/services/ingest_worker.py
polylogue/sources/dispatch.py
polylogue/sources/live/append_ingest.py
polylogue/sources/live/batch.py
polylogue/sources/revision_backfill.py
tests/unit/pipeline/test_resilience.py
tests/unit/sources/test_dispatch_payloads.py
tests/unit/sources/test_live_batch_support.py
tests/unit/sources/test_revision_backfill.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Rebased perf/sharded-from-empty-rebuild onto origin/master (merged since branch point: #3494 deadline_check, #3496 pipeline-decode pin, #3497 conversational-evidence gate, #3498 lineage fixture fix). Conflicts resolved in rebuild_index.py by keeping both #3494's mid-replay deadline checkpointing (non-sharded path) and this PR's shard dispatch, with shard_count>1 + pass_deadline_seconds explicitly rejected (validator and CLI) since the sharded path has no deadline_check seam yet. Ref polylogue-pzxm Co-Authored-By: Claude <noreply@anthropic.com>

…3555) ## Summary Fixes 11 of 48 tests failing under `devtools test tests/unit/storage/test_repair.py -k raw_materialization` on master (polylogue-3jv24). Test-fixture staleness, not a production bug: production code is unchanged. ## Problem PR #3497 wired `require_positive_conversational_evidence` into `sources/revision_backfill.py`, the offline replay path used by raw materialization repair. That filter refuses any parsed session whose messages carry no real text/content blocks -- correct, deliberate behavior, already measured against the live archive and documented at length in `sources/dispatch.py`. #3497 never touched `test_repair.py`, and eleven of its raw-materialization fixtures wrote raw payloads that were structurally valid (a bare Codex `session_meta` line, or a ChatGPT `{"mapping": {"node": {}}}` with an empty node body) but carried zero actual message content -- exactly what the new gate is designed to refuse. Once the gate went live, those sessions were silently dropped during replay, so `repaired_count` / `executed_count` assertions that previously reflected "this raw row got materialized" started coming back short or zero. Confirmed via direct reproduction: e.g. `test_raw_materialization_execute_limits_authority_selection` asserted `repaired_count == 2` and got `0`, with `polylogue-9ykn: refusing session execute-N ... no messages, no positive conversational evidence` in captured stderr for every affected fixture. This is **not** a data-loss regression in raw-authority replay batching (the framing the tracking bead raised as a live possibility) -- it is test-infra staleness. These are repair-plumbing tests (authority selection, batch limits, retries, plan conservation) that only ever cared about selection/batching/retry semantics, never about parse-content semantics, until the content gate started enforcing that distinction. ## Solution Added a minimal, real message to each of the eleven affected fixtures in `tests/unit/storage/test_repair.py`: - Codex-shaped payloads: append one `response_item` line (`role":"user"`, `input_text` content) after the existing `session_meta` line. - ChatGPT-shaped payloads (`test_raw_materialization_split_root_routes_authority_replay`, `test_raw_materialization_uses_authority_replay_not_legacy_batch_parser`): replace the empty `mapping` node body with one carrying a real `message` object. Every touched fixture that cares about byte-size math already overrides `blob_size` via a direct SQL `UPDATE` after the write, so appending message bytes does not perturb any size-based assertion. No non-test file changed. ## Verification - `devtools test tests/unit/storage/test_repair.py -k raw_materialization` -> 48 passed (was 11 failed / 37 passed) - `devtools test tests/unit/storage/test_repair.py` -> 66 passed - `devtools verify --quick` -> exit 0 (ruff format/check, mypy --strict, render all --check, layering, schema-versioning, and the rest all green) Ref polylogue-3jv24  ## Summary by CodeRabbit * **Tests** * Updated storage repair test scenarios to use realistic session and message records. * Improved coverage for replay, authority, scheduling, failure handling, and conservation workflows.  Co-authored-by: Claude <noreply@anthropic.com>

…text Problem: 10 tests in tests/unit/storage/test_raw_authority_ledger.py failed on origin/master (confirmed in a fresh worktree, not testmon-cache noise) -- e.g. test_parsed_timestamp_without_exact_application_receipt_fails_closed asserted RawReplayPlanStatus.TERMINAL but got REJECTED_STALE; test_two_successive_quiescent_censuses_are_required_for_fixed_point asserted repaired_count == 1 but got 0. Root cause (verified via git log/git show, not guessed): PR #3497 (fix(sources): require positive conversational evidence for session creation, closing polylogue-9ykn/polylogue-6mpy, merged 2026-08-01) intentionally added require_positive_conversational_evidence() so a session is only materialized when at least one message carries real text or a content block -- a deliberate, well-documented correctness fix for a live-archive defect (22.4% of sessions were phantom zero-message rows). The ledger test file's `_write_codex_raw` helper predates that gate and defaulted `text=""`, so every one of its raw-authority fixtures now gets refused at parse time ("no messages, no positive conversational evidence") instead of materializing a session -- collapsing repair/census/replay outcomes the tests assert on (repaired_count, plan status, census plan/post-plan counts, application-receipt contents) down to the "nothing materialized" case. These 10 tests exercise raw-authority census/plan/replay/blocker mechanics, not the content-evidence gate itself, so the fix is the test fixture, not production code: give `_write_codex_raw` a non-empty default `text` so its raw writes always produce a materializable session again, matching the new intentional behavior. No production code changed. Also filed polylogue-h7y0j for a sibling break in the same class found while investigating (devtools/raw_authority_scale_proof.py's synthetic codex-session generator hits the same gate) -- confirmed pre-existing on origin/master and out of scope here (different files, not part of this bead), reproduced independently with this fix stashed out. Verification: devtools test tests/unit/storage/test_raw_authority_ledger.py -> 39 passed devtools test -k raw_materialization -> 118 passed devtools test -k raw_authority -> 88 passed, 6 pre-existing failures (all in test_raw_authority_scale_proof.py / test_raw_authority_daemon_health_proof.py, confirmed unrelated: reproduced identically with this commit's diff stashed out; tracked as polylogue-h7y0j) devtools verify --quick -> exit 0 Ref polylogue-k2grh Co-Authored-By: Claude <noreply@anthropic.com>

…text (#3572) ## Summary Fixes 10 failing tests in `tests/unit/storage/test_raw_authority_ledger.py` (confirmed failing on origin/master in a fresh worktree checkout, not testmon-cache noise). All 10 share one root cause and one fix: the test file's `_write_codex_raw` fixture helper defaulted to `text=""`, which a recently-merged, intentional production behavior change now refuses at parse time. Fixing the fixture default (one line) restores all 10. ## Problem `_write_codex_raw`'s default `text=""` predates PR #3497 (`fix(sources): require positive conversational evidence for session creation`, merged 2026-08-01, closing polylogue-9ykn/polylogue-6mpy). That PR added `require_positive_conversational_evidence()`/`message_carries_authored_content()`: a session is only materialized when at least one message carries real text or a content block. It's a deliberate, well-evidenced fix — the live archive had 5,257/23,496 (22.4%) phantom zero-message sessions, and the PR body documents the reconciliation query-by-query. Every one of the 10 failing tests writes a raw codex artifact via `_write_codex_raw` without an explicit `text=`, so its sole message has empty text and gets silently refused ("no messages, no positive conversational evidence") instead of materializing a session. That collapses each test's repair/census/replay/blocker outcome down to "nothing materialized", breaking assertions on `repaired_count`, plan status (`RawReplayPlanStatus.TERMINAL` expected, `REJECTED_STALE` observed), census plan/post-plan counts, and application-receipt contents — none of which are actually about the content-evidence gate; they're about raw-authority census/plan/replay/blocker bookkeeping. ## Solution Per-test investigation (git log/git blame/git show on `polylogue/storage/raw_authority.py` + `polylogue/sources/dispatch.py`, plus the PR #3497 body) confirmed all 10 failures are the same drift, not independent behavior changes: - `test_parsed_timestamp_without_exact_application_receipt_fails_closed` - `test_two_successive_quiescent_censuses_are_required_for_fixed_point` - `test_stale_blocker_resolution_replans_current_evidence_and_resumes` - `test_application_receipt_requires_exact_application_authority[accepted_raw_id / session_id / accepted_content_hash]` - `test_census_ledger_conserves_unselected_plan_and_application_receipt` - `test_frontier_classifies_dangling_head_session_as_corrupt` - `test_frontier_classifies_head_session_raw_mismatch_as_corrupt` - `test_ineligible_quarantined_raw_gets_a_terminal_actuator_not_refine_quarantine` **Verdict for all 10: test-update, not code-fix.** PR #3497's gate is correct and intentional; production code is untouched. `_write_codex_raw`'s default `text` is changed from `""` to `"authored content"` so the fixture always produces a materializable session, matching the new behavior. No test in the file relied on empty-text semantics on purpose (`grep -n 'text=""'` on the file returns nothing). While investigating, found the same class of break in a sibling synthetic-corpus generator, out of scope for this bead (different files, confirmed pre-existing on origin/master independent of this fix — reproduced identically with this diff stashed out): `devtools/raw_authority_scale_proof.py`'s synthetic codex-session generator hits the same gate, breaking 5 tests in `tests/unit/devtools/test_raw_authority_scale_proof.py` and 1 integration test (`tests/integration/test_raw_authority_daemon_health_proof.py::test_real_daemon_drains_backlog_while_staying_probeable`, which times out because the synthetic backlog can never drain). Filed as `polylogue-h7y0j`. ## Verification ``` devtools test tests/unit/storage/test_raw_authority_ledger.py -> 41 passed (2 more tests exist on rebased master than at investigation time; all pass) devtools test -k raw_materialization -> 118 passed devtools test -k raw_authority -> 88 passed, 6 failed — all 6 confirmed pre-existing/unrelated (different files: test_raw_authority_scale_proof.py, test_raw_authority_daemon_health_proof.py; reproduced identically with this branch's diff stashed out; tracked as polylogue-h7y0j) devtools verify --quick -> exit 0 (also ran automatically via the pre-push hook) ``` Ref polylogue-k2grh --------- Co-authored-by: Claude <noreply@anthropic.com>

#3642) ## Summary Fixes the raw-authority-scale-proof synthetic Codex corpus generator so every generated raw payload carries at least one message with real text, and re-derives its minimum-row-bytes accounting from the real header size instead of a guessed constant. ## Problem `devtools/raw_authority_scale_proof.py`'s synthetic codex-session generator wrote the first revision of each logical source as a JSONL payload containing only a `session_meta` record with no message at all. PR #3497 made `require_positive_conversational_evidence` (`sources/dispatch.py`) refuse any session whose sole message carries no real text, and by extension refuse a session that has no message at all. Every generated raw session was refused outright ("no messages, no positive conversational evidence"), which broke 5 tests in `tests/unit/devtools/test_raw_authority_scale_proof.py` and made `tests/integration/test_raw_authority_daemon_health_proof.py::test_real_daemon_drains_backlog_while_staying_probeable` hang for the full 120s timeout, since the synthetic backlog it drives a real daemon against could never actually drain (every candidate refused, forever). Reproduced independently of `polylogue-k2grh`'s `test_raw_authority_ledger.py` fix (stashed that fix out, identical failure), confirming this is a distinct instance of the same PR #3497 fallout class, as `polylogue-h7y0j` describes. ## Solution - Extracted a shared `_payload_header(native_id, revision, *, first)` helper used by both `_write_payload` (the actual JSONL writer) and `_row_sizes` (the byte-budget planner), so the two can never drift out of sync again. The `first=True` (i.e. `previous is None`) header now always includes a `response_item` message with non-empty text alongside `session_meta`. - `_row_sizes`'s per-row minimum-byte floor is now the real cumulative header length for that row (each chained/non-independent revision physically copies the previous row's blob before appending its own new header, so its floor must be the previous row's floor plus its own header, not each row's header length in isolation) instead of the old `256 * (revision + 1)` guessed constant. That constant happened to be generous enough for the old (smaller) header, but undershot the new header size for small scenarios, which silently inflated the generated corpus past the requested `total_payload_bytes` (`SUM(blob_size)` from a manual repro measured 8465 vs. a requested 8192 before this fix, exactly right after fixing it). ## Verification - `devtools test tests/unit/devtools/test_raw_authority_scale_proof.py` — 21 passed (previously 5 failed with `polylogue-9ykn: refusing session ... no messages, no positive conversational evidence`). - `tests/integration/test_raw_authority_daemon_health_proof.py::test_real_daemon_drains_backlog_while_staying_probeable`: with the fix, the synthetic backlog now actually drains (no more refusal warnings in the daemon log, confirmed by grep) instead of hanging for the full 120s timeout as it does on unfixed `origin/master` (reproduced: unfixed run hit `Failed: Timeout (>120.0s) from pytest-timeout`). The test still fails on this host, but on a different, later assertion (`/api/status was unresponsive for ~20-29s, exceeding the documented 5.00s bound`) — this looks like host-load-driven flakiness rather than a regression from this change: the failure duration varies run to run (20.21s, then 29.29s on a second attempt), the daemon log shows no error for `/api/status` at all (just heavy background-thread activity), and this host is currently under substantial contention from concurrent agent worktrees (`uptime` load average ~28 on 24 cores, ~15 GiB swapped at the time of testing). Recommend re-verifying this specific integration test on a quieter host/CI; not something a fixture-text fix should paper over by loosening the responsiveness bound. - `ruff check` / `ruff format --check` / `mypy --strict` clean on the touched file. - `devtools verify --quick` — exit 0. Ref polylogue-h7y0j

Sinity and others added 3 commits July 31, 2026 23:39

Sinity merged commit 0c81414 into master Jul 31, 2026
3 checks passed

Sinity deleted the fix/sources/classification-gate-positive-evidence branch July 31, 2026 22:54

This was referenced Aug 1, 2026

fix(storage): let a raw rewrite its own accepted revision head #3527

Merged

fix(test): add real message content to raw-materialization fixtures #3555

Merged

Sinity mentioned this pull request Aug 2, 2026

test(storage): give raw-authority-ledger codex fixtures real message text #3572

Merged

Sinity mentioned this pull request Aug 3, 2026

fix(devtools): give synthetic codex raw payloads real message text #3642

Merged

Navigation Menu

fix(sources): require positive conversational evidence for session creation - #3497

fix(sources): require positive conversational evidence for session creation#3497
Sinity merged 3 commits into
masterfrom
fix/sources/classification-gate-positive-evidence

Sinity commented Jul 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jul 31, 2026

Uh oh!

coderabbitai Bot commented Jul 31, 2026

Review limit reached

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Sinity commented Jul 31, 2026

Summary

Problem

Solution

polylogue-6mpy

polylogue-9ykn

Reconciliation (AC b) — read-only, ?mode=ro

Anti-vacuity statement

AC matrix

Verification

Residuals / follow-ups

Uh oh!

chatgpt-codex-connector Bot commented Jul 31, 2026

Uh oh!

coderabbitai Bot commented Jul 31, 2026

Review limit reached

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Reconciliation (AC b) — read-only, `?mode=ro`