wire or delete unwired judgment/reference-pipeline/cost primitives - #3430
Merged
Conversation
Problem: the six-tool MCP cutover (#3095, polylogue-t46.8) retired the standalone cost_outlook MCP tool without a replacement. The underlying Polylogue.cost_outlook facade method stayed live and CLI-reachable, but tests/unit/cost/test_contract_suite.py::test_mcp_cost_outlook_tool_uses_shared_envelope was left permanently xfail(strict=False, raises=KeyError) pending a design decision between point-fixing the one tool vs. re-hosting the whole 11-type INSIGHT_REGISTRY cost/usage family generically. Solution: point-fix. get(ref="cost-outlook:<plan_name>", projection=<method>) now resolves through the same Polylogue.cost_outlook() production route the CLI `analyze --cost-outlook` command uses (polylogue/cli/query_verbs.py), including the same PlanLookupError and "no cycle window" degradation via cost_outlook_availability. This avoids adding an 11th top-level tool, which would contradict the deliberate 10-tool consolidated architecture. Re-hosting cost_rollups/session_costs generically remains open design scope, noted in the updated cost_of() MCP prompt guidance rather than silently pointed at tools that no longer exist. Replaced the permanently-xfail test (it could never pass as literally written -- it asserted a distinct "cost_outlook" tool_manager entry that the new design deliberately does not create) with two real-route tests exercising get()'s new cost-outlook resolution: the happy path and the typed invalid_argument error path for an unknown plan. Verification: devtools test tests/unit/cost/test_contract_suite.py polylogue/mcp/server_cutover.py polylogue/mcp/server_prompts.py -> 19 passed. mypy --strict polylogue/mcp/server_cutover.py -> success. ruff check/format clean. Ref polylogue-hg97 Co-Authored-By: Claude <noreply@anthropic.com>
… (polylogue-rxdo.6) Problem: ReferenceQueryPipeline/RefOperand/parse_reference_query_pipeline (archive/query/expression.py) and the real planner seam DurableRefResolver + ArchiveCanonicalPlanEvaluator (landed as production implementations in PR #2899) had zero callers anywhere in polylogue/cli/*.py, polylogue/mcp/*.py, or polylogue/daemon/*.py. compile_expression unconditionally hard-erred on any `from <ref>` pipeline ("requires the reference-aware query planner"). Solution: wire the MCP `query` tool to resolve `from query:<hash>`, `from result-set:<id>`, and `from query-run:<id>` reference pipelines through the real resolver/evaluator seam before falling through to the ordinary DSL path, returning member refs + planner lineage. This is the bounded slice the bead's own 2026-07-14 note named as the concrete next step, deliberately scoped to the MCP query() handler (self-contained, ~80 lines) rather than CLI archive_query.py (2500+ lines, central to every query-mode invocation -- the same note explicitly flagged blind wiring there as reckless). Stage composition after the root operand (`| group by ... | count`) is honestly NOT implemented by this change -- a reference pipeline with stages returns a typed `not_implemented` error naming the gap instead of silently dropping the stages, crashing, or (previously) a generic compile error. Full stage composition, cohort resolution, and CLI `find` wiring remain open scope on polylogue-rxdo.6, noted as such rather than closed. Verification: devtools test tests/unit/mcp/test_reference_query_pipeline.py tests/unit/mcp/test_contract_evidence.py -> 17 passed (3 new tests exercise the real production route: a seeded archive, a real durable query:<hash> object via put_query, and the actual DurableRefResolver/ ArchiveCanonicalPlanEvaluator classes -- no test double). mypy --strict polylogue/mcp/server_cutover.py -> success. ruff check/format clean. devtools render all --check -> OK. Ref polylogue-rxdo.6 Co-Authored-By: Claude <noreply@anthropic.com>
…do.9.1) Problem: MetricDefinition/MetricRegistry (polylogue/insights/measurement/ metric.py, PR #2888 merged, mutation-tested) had zero production callers. The corrective AC's second consumer path ("one hash resolves through both query/analysis and statistical-registry paths") depends on polylogue-9l5.7's statistics registry, which is a separate unstarted epic -- deleting the identity/schema layer was not an option since 9l5.7.2 and polylogue-stc both explicitly depend on MetricDefinition as their foundation, and building the full 9l5.7 composition/aggregation engine to give it a "real" second consumer would be exactly the rxdo-epic scope expansion this pass is supposed to avoid. Solution: the bounded, honest slice available without 9l5.7 is real registry population + resolution. polylogue/insights/measurement/ registered_metrics.py declares one concrete MetricDefinition for an existing, already-computed construct (session cost -- the same mixed provider-reported/catalog-estimated basis cost_outlook already renders), registered in a process-wide DEFAULT_METRIC_REGISTRY. The MCP get() tool now resolves get(ref="metric:session_cost_usd") or get(ref="metric:<hash>") against it. This proves the identity/registry machinery resolves through a real production surface. It does NOT execute the metric (no aggregation engine exists) or attach a metric_ref to any computed value -- both remain open scope, honestly not claimed. Verification: devtools test tests/unit/mcp/test_metric_ref_resolution.py tests/unit/insights/measurement/test_registered_metrics.py -> 6 passed (production-route tests: real MCP get() tool, real registry, no mocks for the registry itself). mypy --strict -> success on all 4 changed/added files. ruff check/format clean. devtools render topology-projection (new module) + render all --check -> OK. Ref polylogue-rxdo.9.1 Co-Authored-By: Claude <noreply@anthropic.com>
…ntrols Problem: three related judgment-mechanism primitives were built, unit- and mutation-tested, but had zero production callers: - blind_items()/BlindingReceipt (rxdo.9.6): called only by ElicitationSession, which itself had zero production callers. - ClaimWithControls (rxdo.9.7): zero callers outside its own test. - compute_calibration() (rxdo.9.12): called only by cascades.py within the same package, itself unreachable from CLI/MCP/daemon. The storage chokepoint they all ultimately feed, upsert_comparative_judgment_assertion / list_comparative_judgments (polylogue/storage/sqlite/archive_tiers/user_write.py), was fully built (idempotent by content hash, reuses the candidate-coercion invariant) but also had zero production callers -- only its own unit test. Full wiring per the mechanisms' own design (the 37t.12/7ome judgment UX epic: inbox, micro-moments, deliberate resorter sessions) is out of scope -- DO NOT expand into the rest of the rxdo epic per this pass's own guardrail. DELETE was rejected: polylogue-7ome's design explicitly plans to reuse BlindedItem/BlindingReceipt/ClaimWithControls/calibration as its rendering primitives, so deleting them would contradict already-designed future scope. Solution: a new `polylogue compare` CLI command (polylogue/cli/commands/ compare.py) is the bounded, real production surface these mechanisms were missing: - `compare --left <ref> --right <ref> --dimension ... --rubric ...` (no --verdict) prints a blind_items()-masked pair -- provenance fields (actor/model/etc, supplied via --left-field/--right-field) hidden, receipted via BlindingReceipt. - The same invocation with --verdict builds a ComparativeJudgment (build_comparative_judgment), persists it through the previously-uncalled upsert_comparative_judgment_assertion (new facade method Polylogue.record_comparative_judgment), and reveals identity via blinding.reveal(). - `compare --calibration --gold-actor <ref>` reads back every recorded judgment (Polylogue.list_comparative_judgments, also newly wired) and reports compute_calibration()'s per-(actor,execution-context,dimension) agreement rate against the gold actor. Separately, FindingAssertion gains an optional `controls` field (rxdo.9.7): a detector/analyst declares a NegativeControl plus its own observed outcome at write time; _finding_value validates each via validate_control() (mutation-tested, reused not re-derived) and fails the whole finding write closed on a confounded/unmatched control rather than storing it silently. Polylogue.resolve_ref's finding-provenance path now constructs ClaimWithControls from the stored controls and renders claim-vs-control together, downgrading rank_tier and adding a caveat when a control failed -- exactly the composition the bead's own corrective note said was "satisfied at the type/rendering-function level but not end-to-end." Verification: devtools test tests/unit/cli/test_compare_command.py tests/unit/storage/test_archive_tiers_assertions.py tests/unit/api/test_facade_contracts.py -> 322 passed (one pre-existing, unrelated real-clock failure confirmed present on unmodified master via git stash: test_archive_tiers_api_raw_artifacts_read_source_tier). Real production-route tests throughout: real CLI invocation via CliRunner against a real archive (cli_workspace fixture), real storage writer, no mocks on the judgment/controls machinery itself. mypy --strict clean on all 7 changed/ added files. ruff check/format clean. devtools render topology-projection (new compare.py module) + render cli-reference (new command) + render all --check -> OK. Ref polylogue-rxdo.9.6 Ref polylogue-rxdo.9.7 Ref polylogue-rxdo.9.12 Co-Authored-By: Claude <noreply@anthropic.com>
Records this session's per-bead disposition notes for the unwired-primitives sweep: hg97 closed (wired via MCP get()); rxdo.6, rxdo.9.1, rxdo.9.6, rxdo.9.7, rxdo.9.12 left open with WIRED/PARTIAL progress notes (real remaining scope in each, not claimed complete); rxdo.9.8 confirmed unstarted, left open untouched per operator instruction not to manufacture an implementation. Co-Authored-By: Claude <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9f1d62b1b5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| rationale=rationale, | ||
| rationale_visible=rationale_visible, | ||
| ) | ||
| envelope = run_coroutine_sync(env.polylogue.record_comparative_judgment(judgment, author_kind="user")) |
There was a problem hiding this comment.
When --actor-ref is an agent:* identity, this hard-coded author_kind="user" bypasses the storage layer's agent-promotion gate and writes the verdict as ACTIVE; list_comparative_judgments() then includes it in calibration as trusted without operator review. Derive the author kind from the validated actor or otherwise prevent non-user actors from taking the user-authored path.
Useful? React with 👍 / 👎.
| dimension=dimension, | ||
| verdict=verdict_enum, | ||
| judge=judge, | ||
| blinded=True, |
There was a problem hiding this comment.
With --verdict, the command never presents the masked pair before recording: the caller supplies the verdict alongside the identity-bearing --left/--right arguments, and the projection uses fixed order [0, 1]. Nevertheless, the persisted judgment claims blinded=True, so provenance-aware analysis and calibration treat an unblinded choice as blinded; this needs a receipt-bound two-step interaction or must be recorded as unblinded.
Useful? React with 👍 / 👎.
| 4. For the current repo's sessions: search(query={repository_query!r}) then session_costs(session_id=<id>) per hit — cost tools have no repo filter. | ||
| 1. get(ref="cost-outlook:<plan_name>") — current billing-cycle projection for a configured subscription plan (burn rate, quota pressure, overage). | ||
| 2. status(scope="archive", include=["provider_usage"]) — usage accounting diagnostics without billing estimates. | ||
| 3. For the current repo's sessions: query(expression={repository_query!r}) for hits, then get(ref="session:<id>") per hit for per-session cost detail. |
There was a problem hiding this comment.
When an agent follows this prompt, get(ref="session:<id>") resolves to SessionSummaryPayload, which contains identity/title/message-count/timestamps but no cost fields, so step 3 cannot provide the promised per-session cost detail. The next line already acknowledges that per-session cost rollups have no MCP surface; direct the workflow to the stated CLI fallback or add an actual cost projection instead.
AGENTS.md reference: AGENTS.md:L351-L353
Useful? React with 👍 / 👎.
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (15)
📝 WalkthroughWalkthroughThe PR adds a ChangesJudgment and reference resolution
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant CompareCLI
participant Archive
participant Calibration
Operator->>CompareCLI: provide item references and optional verdict
CompareCLI->>CompareCLI: blind item records
CompareCLI->>Archive: record comparative judgment
Operator->>CompareCLI: request calibration
CompareCLI->>Archive: list judgments
CompareCLI->>Calibration: compute agreement statistics
Calibration-->>Operator: calibration output
sequenceDiagram
participant MCPClient
participant MCPServer
participant ArchiveResolver
participant MetricRegistry
MCPClient->>MCPServer: query from reference or get typed ref
MCPServer->>ArchiveResolver: resolve query reference
ArchiveResolver-->>MCPServer: lineage and members
MCPServer->>MetricRegistry: resolve metric reference
MetricRegistry-->>MCPServer: metric definition
MCPServer-->>MCPClient: result payload or typed error
Possibly related PRs
Suggested labels: ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
# Conflicts: # .beads/issues.jsonl
# Conflicts: # docs/plans/topology-target.yaml
Sinity
added a commit
that referenced
this pull request
Jul 31, 2026
…ty (#3445) ## Summary Fixes three MCP/CLI query-surface defects found by a surface-coherence audit and a shipped-defaults sweep, ahead of a report that cites these surfaces as evidence: - **polylogue-hnl7 (P1)**: `query()`'s default projection silently dropped `origin`/`tag`/`repo`/`since`/`until`/`min_messages`/`max_messages`/`min_words`, and accepted an unrecognised `origin` without error. - **polylogue-f5tq**: `_archive_facet_buckets(include_deferred=True)` — the shipped default feeding the `facets` verb — was structurally untestable by its only existing test. - **polylogue-il50 (P1)**: 6 of 7 declared MCP prompts instructed callers to invoke tool names retired at the 10-tool cutover; 5 live prompts were undeclared in the other direction; `EXPECTED_PROMPT_NAMES`/`EXPECTED_RESOURCE_URIS` were dead, unreferenced test constants. ## Problem **hnl7**: `query(expression='messages where role:user | count', origin='claude-code-session')` returned 208,061 (the whole-archive count) instead of the origin-scoped count, because `polylogue/mcp/server_cutover.py`'s default-projection dispatch only forwarded `expression`/`limit`/`continuation` to `query_units`, even though `query_units` (`polylogue/api/archive.py:3459`) already accepts every one of these filters as keyword arguments. `origin='bogus-origin'` was accepted silently rather than rejected, where the CLI's `--origin` validator raises. **f5tq**: an AST sweep for untested shipped boolean defaults found `_archive_facet_buckets(..., include_deferred=True)` (the default, and the branch that does the real SQL aggregation) was never exercised — the one existing test builds its archive stub with `_conn=None` and passes `include_deferred=False`, so passing `True` against that stub would crash on the `None` connection. **il50**: `polylogue/mcp/server_prompts.py`'s `resume_context`/`postmortem_last`/`decisions_about`/`unacknowledged_failures`/`sessions_touching_file` prompts (5 of 6 broken; `cost_of` was already fixed by the concurrently-merged #3430) named retired tools (`find_resume_candidates`, `get_session_summary`, `search`, `list_marks`, `blackboard_list`, etc.) in their own call-sequence instructions. An agent following a prompt's own guidance calls a tool that doesn't exist. `analyze_errors`/`summarize_week`/`extract_code`/`compare_sessions`/`extract_patterns` were live-registered but absent from `TARGET_PROMPTS`, leaving completeness/discovery consumers blind to them. ## Solution - **hnl7**: forward `origin`/`tag`/`repo`/`since`/`until`/`min_messages`/`max_messages`/`min_words` to `query_units` for the default projection. Reject unrecognised `origin` tokens loudly against `core.sources.CORE_SCHEMA_ORIGINS` (the same vocabulary the CLI's `--origin` validator uses). Reject `sort` on the default projection (no session-level ordering exists for unit-source rows) instead of silently ignoring it. - **f5tq**: added a real test exercising `_archive_facet_buckets(..., include_deferred=True)` against a seeded `ArchiveStore` connection, asserting the SQL-aggregated families (`role_counts`, `message_types`) are populated. Triaged the sweep's other ~15 findings in the commit body: reproducing the sweep locally reproduces its own noted caveat (bare-name keyword matching is noisy — e.g. it false-flags `run_blob_gc`'s `dry_run=False` default even though a dozen tests exercise it by omitting the kwarg), and manual spot-checks of the bead-named examples (`exclude_none`, `detail`, `require_overlays`, `include_rows`) show serialization/reporting-detail toggles, not a second concrete defect like the facet-buckets case. - **il50**: rewrote the 5 broken prompts' instructions to name only the live 10-tool surface (`context(intent="resume", ...)`, `query(projection=...)`, `status(scope=...)`, `get(ref=...)`). Added the 5 undeclared prompts to `TARGET_PROMPTS`. Made `EXPECTED_PROMPT_NAMES` declaration-derived (mirroring `EXPECTED_TOOL_NAMES`) instead of hand-copied. Removed the dead, doubly-stale `EXPECTED_RESOURCE_URIS`/`EXPECTED_RESOURCE_TEMPLATE_URIS` rather than "fixing" them to match today's registration — `TARGET_RESOURCES` describes an aspirational future surface (polylogue-t46.8.2/t46.8.3) that doesn't match live `server_resources.py` registration, so deriving from it would assert something not yet true; left a pointer comment instead of duplicating that migration here. Added `tests/unit/mcp/test_prompt_registry_pinning.py`: registered-prompts-equal-declared-prompts, and every prompt's rendered text references only live tool names. A follow-on commit drops the internal `migration_owner` bookkeeping field from the `polylogue://capabilities/query` discovery payload's `mcp_algebra` roster — growing `TARGET_PROMPTS` from 7 to 12 pushed that byte-budgeted resource over `MCP_RESPONSE_BUDGET_BYTES`, caught by the existing `test_query_capability_resource_exposes_mcp_algebra_and_valid_terminal_forms`. A last commit regenerates the topology projection for two modules (`polylogue/cli/commands/compare.py`, `polylogue/insights/measurement/registered_metrics.py`) added by the concurrently-merged #3430 without that regeneration — reproduced as pre-existing/unrelated on a bare rebase via an isolated `git worktree add`, but needed to get `devtools verify --quick` (the pre-push gate) green at all. ## Per-bead AC disposition **polylogue-hnl7**: satisfied. Origin-filtered count now matches the CLI; bogus origin now rejected loudly; `sort` on the default projection now rejected loudly instead of silently ignored. **polylogue-f5tq**: `_archive_facet_buckets(include_deferred=True)` satisfied with a real test + anti-vacuity check (inverting the branch condition makes both facet-bucket tests fail). The 17-item sweep triage is satisfied as a documented spot-check rather than 15 individual tests: reproducing the sweep found it structurally noisy (false positive on `run_blob_gc`), and the bead-named examples inspected are cosmetic-branch toggles, not confirmed second defects. No `devtools lab policy` gate was added, per the operator's standing no-completeness-check-theater rule — this pass did not surface a second migratable defect to justify one. **polylogue-il50**: satisfied both directions — declared prompts now name only live tools (5 rewritten + `cost_of` already fixed upstream), the 5 undeclared prompts are now declared, and both directions are pinned by a real test instead of a dead constant. The `EXPECTED_RESOURCE_URIS` cross-reference is intentionally *not* duplicated here (per the bead's own "cross-reference, do not duplicate" framing for `polylogue-t46.8.2`); the dead constant was removed with a pointer to the real gap rather than force-fit to today's registration. ## Verification - `devtools test tests/unit/mcp/test_query_default_projection_filters.py` → 3 passed. - `devtools test tests/unit/mcp/test_query_gap_projections.py tests/unit/mcp/test_query_request_contracts.py tests/unit/mcp/test_bounded_query_transport.py` → 22 passed. - `devtools test tests/unit/api/test_facade_contracts.py -k facet_buckets` → 2 passed. - `devtools test tests/unit/mcp/test_prompt_registry_pinning.py tests/unit/mcp/test_prompt_query_parity.py` → 14 passed. - `devtools test tests/unit/mcp/ tests/unit/api/test_facade_contracts.py tests/unit/agent_integration/` → 569 passed, 1 pre-existing unrelated real-clock failure (`test_archive_tiers_api_raw_artifacts_read_source_tier`, explicitly documented as pre-existing on unmodified master in the concurrently-merged #3430's own commit message). - `devtools render all --check` → clean (grepped for "out of sync"). - `devtools verify --quick` → exit 0, all 18 steps ok. - Live archive (`/realm/db/polylogue`, read-only, in-process `build_server()`): `origin="claude-code-session"` → 141,652 (CLI `--origin claude-code-session find "messages where role:user | count"` → 141,651; off-by-one is archive growth between the two calls); no filter → 208,061 (the old wrong answer, now scoped correctly); `origin="bogus-origin"` → `{"ok": false, "code": "invalid_argument", ...}`. - Anti-vacuity checks performed and reverted before commit for each fix (documented per-commit): inverting `_archive_facet_buckets`'s branch condition, reverting `decisions_about`'s query fix to the retired `search()` call, deleting an `analyze_errors` `TARGET_PROMPTS` entry — all three make the corresponding new test fail with the expected error. ## Not run - `devtools verify --all` (full non-integration suite) — the touched-surface `devtools test` runs above cover the changed modules; a full run wasn't judged necessary for this scope. --------- Co-authored-by: Claude <noreply@anthropic.com>
Sinity
added a commit
that referenced
this pull request
Jul 31, 2026
Problem: `devtools verify topology` blocked (2 orphans: polylogue/cli/commands/compare.py, polylogue/insights/measurement/registered_metrics.py) on origin/master already, from dd912a7 ("wire or delete unwired judgment/reference-pipeline/cost primitives", #3430) adding those modules without regenerating the projection. Confirmed pre-existing and unrelated to the reasoning-content-capture fix on this branch (stashed that diff and reproduced the same failure against a clean checkout). The pre-push verify gate blocks on it regardless of blame, so folding the regen in here rather than leaving push blocked. Solution: `devtools render topology-projection` picks up both files under their existing owners; no manual edits. Verification: devtools verify topology now reports realized=1103 declared=1103 blocking=False (was declared=1101, blocking=True). Co-Authored-By: Claude <noreply@anthropic.com>
Sinity
added a commit
that referenced
this pull request
Jul 31, 2026
…ns (#3447) ## Summary Reasoning/thinking content was invisible in the archive on both coding origins, via two independent mechanisms. Both are fixed here: Claude Code's empty-body thinking blocks are now recorded instead of dropped, and Codex's `reasoning` records are now materialized as real content instead of being read by nothing at all. ## Problem **The loss is shaped like a finding, not a gap.** Live archive query, `blocks.block_type='thinking'` grouped by month for `claude-code-session`: | month | sessions | thinking_blocks | |---|---|---| | 2026-07 | 1,088 | **0** | | 2026-06 | 788 | **0** | | 2026-05 | 2,107 | 50,394 | | 2026-04 | 854 | 9,859 | | 2026-03 | 2,357 | 4,433 | | 2026-02 | 1,687 | 8,505 | | 2026-01 | 1,609 | 24,813 | An analyst reading this would conclude "reasoning declined sharply after May 2026" — confidently, and completely falsely. The model kept reasoning; the archive stopped recording it. ### Defect A — Claude Code: empty-body thinking blocks dropped entirely `polylogue/sources/parsers/base_support.py`'s `content_blocks_from_segments` had `if text:` with no `else` around THINKING segment handling. Since roughly 2026-06 the wire ships thinking blocks with an **empty** `thinking` body and a `signature` only — verified directly against raw `~/.claude/projects/-realm-project-polylogue/*.jsonl` across dates: | date sampled | thinking records | empty-body | non-empty | |---|---|---|---| | 2026-02-11 | 8, 68 | 0 | 8, 68 | | 2026-07-29 | 2,300 | 2,300 | 0 | | 2026-07-30 | 1,010, 275 | 1,010, 275 | 0 | | 2026-07-31 | 112, 499 | 112, 499 | 0 | Ground-truth sessions named in scope, live archive (read-only query, before this fix): | session | raw thinking blocks (all empty-body+signature) | archived thinking blocks | thinking_count | |---|---|---|---| | `38baa1de-9715-48fa-8175-f2a29d92800e` | 499 | 0 | 0 | | `53e64853-1793-43d2-80ac-a41a8c5a56a2` | 275 | 0 | 0 | The reasoning text is genuinely absent from the wire since ~2026-06 (Anthropic API/CLI behavior change, not a parser blind spot) — but the *fact that the model reasoned* is not absent, and that fact is exactly what the `if text:` guard destroyed. ### Defect B — Codex: reasoning records read by nothing `polylogue/sources/parsers/codex.py`'s `_compact_response_payload` (the generic session_event compactor for `response_item`/`event_msg` records) has no branch for `type: "reasoning"`. Neither `summary` nor `content` is a recognized key anywhere in the compactor, so a reasoning record's session_event payload is **only** `{"source_index": N, "type": "reasoning"}` — measured directly: ```python >>> _compact_response_payload({"type": "reasoning", "summary": [...], "content": None, "encrypted_content": "..."}, index=1) {'source_index': 1, 'type': 'reasoning'} ``` No message, no block, nothing FTS-reachable was ever produced for a `reasoning` record. Full corpus scan of this operator's local Codex sessions (`~/.codex/sessions/**/*.jsonl`, 3,213 rollout files): | metric | count | % | |---|---|---| | total `reasoning` records | 1,182,071 | — | | records with recoverable `summary` text | 285,985 | 24.2% | | records with non-null `content` | 0 | 0% | | rollouts containing ≥1 reasoning record | 2,961 / 3,213 | 92.2% | | rollouts with ≥1 recoverable summary | 863 / 3,213 | 26.9% | `content` (the full trace) is essentially always null on the wire — Codex encrypts it into `encrypted_content` instead, which this archive cannot decrypt and does not attempt to store. ### Per-origin reasoning/thinking capture audit | Origin | Status | Evidence | |---|---|---| | Claude Code | **Fixed here** (was: dropped when empty-body) | `base_support.py` `content_blocks_from_segments`, guard removed | | Codex | **Fixed here** (was: never read at all) | `codex.py`, new `_codex_reasoning_message` | | ChatGPT | Captured | `chatgpt.py:727-737`: `content_type in ("thoughts", "reasoning_recap")` -> `BlockType.THINKING`, no emptiness guard observed | | Gemini / aistudio-drive | Captured, with a caveat | `gemini_message.py:271-274`: `raw_part.get("thought") is True` -> `ContentType.THINKING`, but gated by `if part_text:` -- the same empty-body shape as Defect A is structurally possible here and untested against real emptied-thinking Gemini payloads. Not observed to occur in this operator's corpus; flagged as a follow-up audit item, out of scope for this PR (Gemini thinking-signature capture, `thoughtSignatures`, is a separate, already-partially-modeled construct in `drive_support_blocks.py`) | | Hermes | Captured | `hermes_state.py:705-709`: `reasoning_content`/`reasoning` columns -> `BlockType.THINKING` unconditionally when present | | Antigravity | Unknown / not observed | No `thinking`/`reasoning` reference anywhere in `antigravity.py`; the markdown-export/brain-metadata ingestion path may simply have no reasoning construct on the wire for this product, or this is genuinely unaudited. Flagged as a follow-up, not fixed here (no reproducing evidence available in this operator's corpus to establish which) | ## Solution - **`polylogue/sources/parsers/base_support.py`**: `content_blocks_from_segments` now always appends a THINKING block for `type: "thinking"` segments -- `text=None` when the wire carries none (rather than `""`, matching how other block types signal "no text"), `signature` captured when present. - **`polylogue/sources/parsers/base_models.py`**: `ParsedContentBlock` gains a `signature: str | None` field -- the provider-issued cryptographic attestation for a THINKING block (Claude's extended-thinking signature; Gemini's `thoughtSignatures` are the same construct under a different name). - **`polylogue/storage/sqlite/archive_tiers/index.py` / `archive_tiers_specs.py` / `write.py`**: `blocks.signature` (nullable TEXT) added to the DDL, column spec, and row-builder. Deliberately **excluded** from `_block_content_hash` and the lineage prefix signature (`_message_signature_from_blocks`) -- providers re-sign on every replay, so including it would break citation-anchor and fork-prefix matching for otherwise-identical replayed content across a session fork/resume. `INDEX_SCHEMA_VERSION` bumped 48->49. - **`polylogue/storage/sqlite/lifecycle.py`**: v49 `IndexDeltaDeclaration` with `SEMANTIC_REPARSE` -- the same v42/v44/v45/v46/v48 "values depend on parser semantics, no clone-safe SQL delta" shape (the new column is additive/clone-safe on its own, but recovering the previously-dropped/discarded historical thinking/reasoning *content* requires re-parsing raw evidence, which a shape-only fast-forward cannot do). - **`polylogue/sources/parsers/codex.py`**: new `_codex_reasoning_message`/`_codex_reasoning_joined_text` materialize a standalone `reasoning` response_item as a `MessageType.THINKING` message (`role=ASSISTANT`, `material_origin=ASSISTANT_AUTHORED`) with one `BlockType.THINKING` block per recovered text source (`summary`, then `content` if distinct); when neither carries text, a single block with `text=None` still records that reasoning occurred. Wired into the existing `response_item`/`event_msg` dispatch loop alongside `_codex_tool_message`/`_codex_event_message`. Deliberately uses `BlockType.THINKING` (not the vocabulary's `BlockType.REASONING`), matching every other origin's existing convention (ChatGPT, Gemini, Hermes, local_agent) -- `BlockType.REASONING` is presently unused anywhere in the codebase; introducing it here would fragment the `thinking_count` aggregate instead of feeding it. - **Read-path wiring** (`polylogue/storage/runtime/archive/records.py`, `polylogue/storage/hydrators.py`, `polylogue/storage/sqlite/queries/mappers_archive.py`, `polylogue/storage/sqlite/queries/attachment_blocks.py`): `signature` threaded through `BlockRecord`, the domain `Message.blocks` dict projection, and the SQL row mapper/SELECT list, so it's actually readable, not write-only. ### Why `blocks.signature` instead of stuffing it into `metadata` `ParsedContentBlock.metadata` exists but is **not** persisted generically to the `blocks` table -- only `metadata.language` is read out of it (`_block_language`, write.py). Storing the signature there would look captured at the parser layer while silently vanishing at write time, which would misstate this PR's own claim. A real nullable column is the honest choice, matching the exact precedent `tool_result_outcome_unknown_reason` (v46) set for provenance-only, hash-excluded columns. ## Compatibility / rebuild `index.db` is rebuildable derived state. This PR's `SEMANTIC_REPARSE` declaration means: - **Recovers automatically going forward**: every session ingested *after* this deploys gets full thinking/reasoning capture (empty-body Claude Code blocks with signature; Codex reasoning summary/content). - **Does NOT recover historical rows in place**: `blocks.signature` will fast-forward to NULL on existing rows, and existing sessions keep their current (wrong) `thinking_count` of 0, until re-parsed. - **Recovering history requires** `polylogue ops reset --index && polylogued run` against the raw-evidence-backed `source.db` (raw JSONL is retained; nothing here touches raw acquisition). **Not executed by this PR** -- the live archive at `/realm/db/polylogue` is read-only from this branch's perspective; a full index rebuild is the operator's call to schedule. - What a rebuild recovers: the *fact* that reasoning occurred (block existence, `thinking_count`, signature where the wire carries one) for every already-acquired session. What it does **not** recover: reasoning *text* for Claude Code sessions where the wire itself never carried text (that data was never on the wire to begin with -- genuinely, permanently gone upstream, confirmed by the Feb-vs-Jun/Jul empty/non-empty split above) and for the ~76% of Codex reasoning records with neither summary nor content text. ## Also fixed (pre-existing, unrelated) `devtools verify topology` was already failing on origin/master (2 orphans: `polylogue/cli/commands/compare.py`, `polylogue/insights/measurement/registered_metrics.py`, introduced by #3430 without a topology-projection regen) -- confirmed via a stash-and-reproduce check against a clean checkout before touching anything here. Folded a `devtools render topology-projection` regen into a separate commit on this branch since the pre-push gate blocks on it regardless of blame. ## Verification ``` ruff format --check / ruff check # all touched files: clean mypy (--strict via project config) # all touched files: Success, no issues devtools test tests/unit/sources/test_parsers_base.py tests/unit/sources/test_parsers_codex.py \ tests/unit/storage/test_column_spec_reordering.py tests/unit/storage/test_archive_tiers_ddl.py \ tests/unit/storage/test_index_fast_forward_lifecycle.py tests/unit/storage/test_schema_policy_contracts.py \ tests/unit/core/test_models.py tests/unit/storage/test_archive_tiers_write.py \ tests/unit/surfaces/test_message_render_envelope.py # all pass; 5 new tests added from real wire shapes (2 Claude Code empty-body/signature # cases, 3 Codex reasoning cases: summary-only, encrypted-only, summary+content); # 1 pre-existing exact-dict assertion updated for the new `signature` key devtools lab policy schema-versioning # Schema evolution policy intact devtools verify --quick # exit_code: 0 (clean, after the topology regen commit) ``` Not run: `devtools verify --all` (full non-integration suite) -- the touched-file testmon selection plus the explicit DDL/schema/parser suites above cover every changed surface; not re-running the full suite per the repo's stated verification cadence. No live-archive writes were made anywhere in this work -- all archive queries used `file:/realm/db/polylogue/index.db?mode=ro`. --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five reviewers independently flagged primitives that were built, unit-tested, and never wired to a production caller. Per-bead disposition below: WIRE (real caller + real production-route test) or DELETE, no third state, plus one confirmed-unstarted bead left honestly open.
Problem
polylogue-hg97:cost_outlookMCP tool retired by the six-tool cutover, replacement design left undecided; contract test permanently xfail.polylogue-rxdo.6:ReferenceQueryPipeline/RefOperand/the realDurableRefResolver+ArchiveCanonicalPlanEvaluatorplanner seam had zero callers in CLI/MCP/daemon;compile_expressionhard-erred on anyfrom <ref>pipeline.polylogue-rxdo.9.1:MetricDefinition/MetricRegistry(content-addressedmetric:<hash>identity) had zero production callers; its designed second consumer (polylogue-9l5.7statistics registry) is a separate unstarted epic.polylogue-rxdo.9.6/.9.7/.9.12:blind_items,ClaimWithControls, andcompute_calibration(comparative-judgment mechanisms) were correctly implemented and mutation-tested but unreachable outside their own tests — and the storage chokepoint they all feed (upsert_comparative_judgment_assertion/list_comparative_judgments) also had zero production callers.polylogue-rxdo.9.8: confirmed never-started (zeroanalysis_recipes/analysis_runscode exists), blocked on an undeclaredpolylogue-60i5migration window.Solution
get(ref="cost-outlook:<plan>")resolves through the realPolylogue.cost_outlookroute (same as CLIanalyze --cost-outlook). Replaced the permanently-xfail contract test with two real production-route tests (avoids adding an 11th top-level MCP tool, which would contradict the deliberate 10-tool consolidation).query()resolvesfrom query:<hash>/result-set:<id>/query-run:<id>through the real resolver/evaluator seam, returning member refs + lineage. Deliberately scoped to the self-contained MCP handler, notcli/archive_query.py(2500+ lines, explicitly flagged as reckless to touch blind by a prior pass). Stage composition (| group by ... | count) returns a typednot_implementederror rather than silently dropping or crashing.get(ref="metric:<hash-or-name>")resolves one real registeredMetricDefinition(session cost) from a new process-wide registry. Identity/resolution only — no aggregation engine (that's9l5.7, out of scope). DELETE was rejected:9l5.7.2/polylogue-stcboth depend on this identity existing.polylogue compareCLI command. Without--verdict, prints ablind_items()-masked pair (provenance hidden, receipted). With--verdict, records aComparativeJudgmentthrough the previously-uncalled storage chokepoint and reveals identity.--calibration --gold-actor <ref>reportscompute_calibration()'s real per-actor agreement. Separately,FindingAssertiongained an optionalcontrolsfield;validate_control()rejects confounded declarations at write time (fail closed), andPolylogue.resolve_ref's finding path now rendersClaimWithControls(downgrade tier + caveat) for real.Verification
devtools test tests/unit/cost/test_contract_suite.py tests/unit/mcp/test_reference_query_pipeline.py tests/unit/mcp/test_metric_ref_resolution.py tests/unit/insights/measurement/test_registered_metrics.py tests/unit/cli/test_compare_command.py tests/unit/storage/test_archive_tiers_assertions.py tests/unit/api/test_facade_contracts.py→ all pass except one pre-existing, unrelated real-clock failure confirmed present on unmodified master (test_archive_tiers_api_raw_artifacts_read_source_tier, verified viagit stash).mypy --strictclean on every changed/added file.ruff check/formatclean.devtools render topology-projection(new modules) +devtools render cli-reference(newcomparecommand) +devtools render all --check→ OK.devtools verify --quick→ exit 0.Per-bead disposition
Ref polylogue-hg97, Ref polylogue-rxdo.6, Ref polylogue-rxdo.9.1, Ref polylogue-rxdo.9.6, Ref polylogue-rxdo.9.7, Ref polylogue-rxdo.9.12, Ref polylogue-rxdo.9.8
🤖 Generated with Claude Code
Summary by CodeRabbit
compareCLI command for blind pairwise judgments, verdict recording, and calibration reports.compareCLI command and updated cost-reporting guidance.