Skip to content

feat(sources): complete cgfy's remaining unread Claude Code wire keys - #3465

Merged
Sinity merged 3 commits into
masterfrom
feature/sources/wire-remaining-claude-code-message-usage-keys
Jul 31, 2026
Merged

feat(sources): complete cgfy's remaining unread Claude Code wire keys#3465
Sinity merged 3 commits into
masterfrom
feature/sources/wire-remaining-claude-code-message-usage-keys

Conversation

@Sinity

@Sinity Sinity commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Summary

Closes out the remainder of polylogue-cgfy's disposition work now that PR #3442 landed the flagship consumption gap (structuredPatch/originalFile/oldString -> file_edits, reachable via API + MCP + CLI read --view file-edits). Adds the two remaining genuinely-unread, real-value wire keys (requestId, thinkingMetadata.maxThinkingTokens) and records the full disposition (read / measured-negative / deliberately-dropped-duplicate) for the rest of the bead's "other unread keys of substance" list in the Claude Code OriginSpec's fidelity_notes, satisfying AC1.

Problem

polylogue-cgfy's 2026-07-29 corpus enumeration found 34 of the 70 most common Claude Code wire keys with zero references in polylogue/sources/. The structuredPatch/originalFile/oldString cluster (105K/92K/86K rows) and most of the "other keys of substance" list (stop_reason, stop_sequence, cache_creation, ttftMs, todos, toolUseResult.sandbox/filenames/numFiles, slug) were captured by prior sessions on this bead and PR #3442, but two items remained genuinely unread with no recorded disposition either way: requestId (1,171 sampled occurrences -- the Anthropic API's per-call request id) and thinkingMetadata.maxThinkingTokens (34 occurrences -- the extended-thinking token budget). The bead's remaining lower-value items (userType, sourceToolAssistantUUID, hookCount/hookInfos, toolUseID) also had no recorded disposition, so AC1 ("every key... classified... recorded in the Claude Code OriginSpec fidelity declaration") was incomplete.

Verified each of those lower-value items directly against a real ~/.claude/projects corpus before classifying:

  • userType is the literal string "external" on every sampled record (measured negative, same class as the bead's own usage.service_tier finding).
  • sourceToolAssistantUUID equals that same record's own parentUuid, already captured as ParsedMessage.parent_message_provider_id -- a duplicate spelling, not new evidence.
  • hookCount/hookInfos describe hook firings already tracked durably (with outcome, which hookInfos lacks) in source.db's raw_hook_events.
  • toolUseID is already consumed for its real signal (the progress/agent_progress delegation-edge disposition documented in code_parser.py's module docstring).

Also discovered en route: master HEAD (5798b3d) failed to import at all (ImportError: cannot import name 'literal_check') -- #3458 deleted it as a supposed zero-call-site dead shim, but #3451 had wired real call sites into it earlier the same day. Fixed and merged separately as #3464 so this branch could even run tests; this branch is rebased on top of that fix.

Solution

  • polylogue/sources/parsers/claude/code_parser.py: _message_usage_event_payload gains a record parameter (the top-level record item, since requestId/thinkingMetadata live there, not inside message); the sole call site now passes record=item. Both new keys ride the existing message_usage session-event payload as request_id/max_thinking_tokens.
  • Production callers exercising the new fields: Session.session_events -> CLI read --view events (polylogue/cli/read_views/events.py -> run_session_events, polylogue/cli/messages.py) and MCP get(ref, projection="events") (polylogue/mcp/server_cutover.py:905) already render every session_events row generically regardless of event_type -- no new consumer code needed, verified by reading both call chains before writing this PR (the generic events projection was wired by prior work, not this PR).
  • polylogue/sources/origin_specs.py: new fidelity_notes entry on the Claude Code OriginSpec recording the full disposition table above.

Verification

  • devtools test tests/unit/sources/test_claude_code_unread_wire_fields.py tests/unit/sources/test_parsers_claude_code_artifacts.py tests/unit/sources/test_origin_specs.py -> 57 passed
  • devtools verify --quick -> exit 0 (format, lint, mypy, render all --check, schema-versioning policy)
  • Anti-vacuity: each new field has a paired "field absent" test asserting the key is omitted from the payload, not fabricated

Ref polylogue-cgfy

Summary by CodeRabbit

  • Enhancements

    • Claude Code usage events now capture available request identifiers and maximum thinking-token settings.
    • Missing metadata continues to be omitted, keeping usage records clean and accurate.
  • Quality Improvements

    • Added coverage to verify that these Claude Code fields are preserved and handled consistently.

Sinity added 2 commits July 31, 2026 16:34
Problem: polylogue-cgfy's corpus enumeration found requestId (1,171 sampled
occurrences) and thinkingMetadata.maxThinkingTokens (34 occurrences) among
the wire keys never read anywhere in polylogue/sources/. requestId is the
Anthropic API's per-call correlation id (distinct from Claude Code's own
record uuid or a tool call's own id) -- real cross-reference value against
provider-side billing/support records. thinkingMetadata.maxThinkingTokens is
the extended-thinking token budget configured for the turn, a reasoning-
effort signal distinct from the actual token counts already captured.

Solution: both are top-level fields on the raw record (not inside
`message`), so `_message_usage_event_payload` gains a `record` parameter and
the sole call site (the existing `message_usage` session-event append) now
passes `item`. Reachable immediately through the existing generic
`message_usage` session-event surface with no new consumer code needed:
`Session.session_events` -> CLI `read --view events` -> `run_session_events`
(polylogue/cli/messages.py) and MCP `get(ref, projection="events")`
(polylogue/mcp/server_cutover.py:905) already render every session_events
row regardless of event_type.

Verification: devtools test tests/unit/sources/test_claude_code_unread_wire_fields.py
tests/unit/sources/test_parsers_claude_code_artifacts.py -> 13 + 38 passed.
Anti-vacuity: each new field has a paired "absent" test asserting the key is
omitted, not fabricated as None/empty.
Problem: polylogue-cgfy AC1 requires every key in its corpus enumeration
classified read / deliberately-dropped-with-reason / to-acquire, recorded in
the Claude Code OriginSpec fidelity declaration. The structuredPatch/
file_edits cluster and the message.stop_reason/cache_creation/todos cluster
were already classified in a prior fidelity_notes entry; the bead's "other
unread keys of substance" list (slug aside, already covered) had no
disposition recorded anywhere.

Solution: added a fidelity_notes entry to the Claude Code OriginSpec
covering requestId/thinkingMetadata (READ, this branch's prior commit),
userType (MEASURED NEGATIVE -- reconfirmed constant "external" against a
second live corpus), sourceToolAssistantUUID (DROPPED, verified equal to
the already-captured parentUuid), hookCount/hookInfos (DROPPED, a less
complete duplicate of source.db's raw_hook_events), and toolUseID (already
consumed via the documented claude_delegation_progress disposition, not a
bare unread field).

Verification: devtools test tests/unit/sources/test_origin_specs.py -> 15
passed.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@Sinity, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 51 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 54a0ca54-2a9a-42a0-af83-55f3ada86ca2

📥 Commits

Reviewing files that changed from the base of the PR and between 57f46b9 and 6011d92.

📒 Files selected for processing (2)
  • polylogue/sources/parsers/claude/code_parser.py
  • tests/unit/sources/test_claude_code_unread_wire_fields.py
📝 Walkthrough

Walkthrough

The Claude Code parser now captures non-empty requestId and valid thinkingMetadata.maxThinkingTokens values in message_usage events. Fidelity documentation and parser contract tests cover the new fields and omission behavior.

Changes

Claude Code usage metadata

Layer / File(s) Summary
Usage metadata extraction and wiring
polylogue/sources/origin_specs.py, polylogue/sources/parsers/claude/code_parser.py
The usage payload builder receives the source record and adds request_id and max_thinking_tokens when valid values are present. Fidelity notes document included and excluded fields.
Usage metadata contract tests
tests/unit/sources/test_claude_code_unread_wire_fields.py
Tests verify preservation and omission behavior for requestId and thinkingMetadata.maxThinkingTokens.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • Sinity/polylogue#2486: Both changes extend _message_usage_event_payload with Claude Code usage metadata.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes completing support for the remaining unread Claude Code wire keys.
Description check ✅ Passed The description covers the summary, problem, solution, verification, and risks sections with specific implementation and test details.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/sources/wire-remaining-claude-code-message-usage-keys

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@polylogue/sources/parsers/claude/code_parser.py`:
- Around line 754-758: Update the maxThinkingTokens validation in the
thinking_metadata handling to accept only non-negative integers, while
continuing to exclude booleans; omit payload["max_thinking_tokens"] for negative
values. Add a -1 fixture and assert that the field is omitted.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e8df8a4d-4cbc-4367-8202-7564c52fe51c

📥 Commits

Reviewing files that changed from the base of the PR and between aeea9c4 and 57f46b9.

📒 Files selected for processing (3)
  • polylogue/sources/origin_specs.py
  • polylogue/sources/parsers/claude/code_parser.py
  • tests/unit/sources/test_claude_code_unread_wire_fields.py

Comment thread polylogue/sources/parsers/claude/code_parser.py
CodeRabbit review on PR #3465: a negative thinkingMetadata.maxThinkingTokens
is not a real token budget and should be omitted rather than persisted.
Adds the >= 0 guard plus a -1 fixture asserting omission.

Verification: devtools test tests/unit/sources/test_claude_code_unread_wire_fields.py
-> 14 passed.
@Sinity
Sinity merged commit 78ea7ed into master Jul 31, 2026
3 checks passed
@Sinity
Sinity deleted the feature/sources/wire-remaining-claude-code-message-usage-keys branch July 31, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant