diff --git a/CHANGELOG.md b/CHANGELOG.md index 03b6986..64dc695 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,31 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ## [Unreleased] +### Added +- **`ccx related [session]`: which sessions connect to this one, and how.** Sessions were islands; the joins were in the transcripts but nothing computed them. `related` derives, deterministically and with evidence, the anchor session's connections to the other sessions of its workspace: `forked_from`/`fork_of` (the transcripts share message ids — Claude Code fork and `ccx fork` copy history verbatim), `mentions`/`mentioned_by` (a session id named in conversation text, quoted), `handoff_from`/`handoff_to` (a baton file — HANDOFF.md, handoffs/, devlog, PLAN.md — written by one and read by the other later), `builds_on`/`built_on_by` (a workspace file edited by one, then read or edited by the other), `overlaps` (concurrent), `previous`/`next`. Strength is a band (strong/medium/weak), never a score; path lists are capped with the count kept and `truncated` set; `--json` (`ccx.related.v1`) carries message id, time, path, and quote per relation. `ccx trace --full` gains the same list as `related`. Design: docs/design/0006-session-connections.md. +- **`ccx search -w/--word` matches whole words.** Matching was substring-only, so a term that prefixes a common word was unanswerable: `search --content semantica` returned 47 sessions, 46 of them "semantic*ally*", and ccx alone could not tell 0 real hits from 46 (docs/devlog/2026-08-18-search-word-boundary-dogfood.org). `-w` demands an ASCII word boundary on each side of the query that starts/ends with a word character (so "semantica-agi" and "(semantica)" still hit; CJK queries are unaffected) and applies to names, summaries, conversation text, and `--raw` lines alike. +- **`ccx search --hits` turns matches into citations.** One row per matching message — time, session, role, message id, quote — oldest first across sessions, `-n`-capped with a visible "showing N of M". The anchors are the same ones `trace` and `view` use, so a claim built on a search can point at its evidence (design: docs/design/0005-evidence-citations-lessons-from-semantica.md). Under `--raw` the unit is a transcript line, anchored by its own `uuid`/`type`/`timestamp`. +- **`ccx search --content` reports when: `FIRST` column, `first_hit` in `--json`, `--sort first|last|hits`.** "When did we first mention X" needs the earliest matching message and an oldest-first order; results only carried session end time and ranked by hit count. Each content hit now records the timestamp of its earliest matching message (parsed messages by default; the raw line's top-level `timestamp` under `--raw`), printed as `FIRST` and sortable with `--sort first`; `--sort last` orders by session activity; `--sort hits` is the old order and the default. +- **Codex interruptions count too.** `event_msg.turn_aborted` with reason `interrupted` (144 in one store) is Codex's "[Request interrupted by user]"; it now parses as the same `interrupt` kind, so `trace` interrupt counts and `log --kind interrupt` cover both providers. +- **Human interventions are first-class in `trace` and `log`.** "[Request interrupted by user]" (the human pressed stop) and permission-prompt rejections ("The user doesn't want to proceed with this tool use") are the human in the loop, but ccx read the first as a *prompt* — it opened a fake turn `u: [Request interrupted by user for tool use]` — and the second as an ordinary tool error. New parser kind `interrupt` (harness marker, never an exchange anchor) and `parser.IsToolDenial`; `trace` counts `interrupts` and `denials` on turn, step, and stats, marks the rejected call `denied` (not an error, it did not run), and badges them in the outline header and rows (`1 interrupt, 2 denied`; step `[4t 1! 1d]`); `log` reports kinds `interrupt` and `tool_denied` (`ccx log --scope today --all --kind interrupt,tool_denied`). 650 interruptions across 321 sessions and 145 rejections in one real store were invisible before. +- **`GET /api/related//` serves session connections as JSON** — the `ccx.related.v1` envelope (`related`, `total`, `shown`, `warnings`; `?limit=N`), computed by the same `trace.RelateWorkspace` the CLI uses, so a web panel or an agent reading the API sees exactly what `ccx related --json` prints. Fetched on demand: it costs a parse of every workspace session (cached after the first call). +- **`ccx search --content` scans prompt history.** Claude Code and Codex append every human prompt to `history.jsonl`, and those files outlive session cleanup (one real store: prompts back to 2025-09-28, sessions back to 2025-12-07). Matches surface as type `prompt` — only for prompts whose session file is no longer in the store, so a prompt never appears twice — with `FIRST`, a `[user]` quote, and under `--hits` a `history:line` anchor. "When did we first say X" now reaches past the session horizon. +- **`ccx view --at MESSAGE_ID [--context N]` walks from a citation to its context.** Search `--hits` and `trace` hand out message ids, but nothing in the CLI could open one; drill-down meant the web page or raw grep (open since docs/devlog/2026-08-03-content-search-noise.org finding 4). `--at` renders the cited message with N messages before and after it (wire order, flattened; the target survives `--brief`), and says where it sits: `message 1 of 763`. Prefixes resolve; ambiguous prefixes are an error, not a guess. + +### Changed +- **`ccx trace` labels narration-less steps.** A step the agent never narrated (straight to tools) rendered as a bare badge row `1. [4t]`; the outline now says what it ran: `(no narration) Bash x3, Read` (docs/devlog/2026-08-17-codex-0147-rollout-drift.org finding 3). +- **`ccx log --kind` and `--match PHRASE [-w]` turn the firehose into a timeline.** `log` emitted every record in scope (16k for one day) with no way to narrow; "what did the humans ask today, across every session" was not answerable. `--kind user_prompt,assistant_message` keeps only those kinds; `--match` keeps records whose raw transcript line contains the phrase (grep parity, `-w` for whole words) — the time-bounded complement of `search --hits`. Metrics stay honest: `records` is scope-wide, new `records_matched` is the narrowed count, `showing` is after `-n`. +- **`ccx search --content` is ~10x faster and shows progress.** The scan ran on one core and lowercased every transcript line: 6m18s cold / 1m14s warm over a 3.5 GB store, silent throughout. Sessions now scan on a bounded worker pool (up to 8), the raw prefilter matches case-insensitively without allocating and stops at the first hit, and a `scanning N/M sessions` line ticks on stderr when it is a terminal. Same store, warm: 7.6s. + +### Fixed +- **`ccx log` now applies the same conversation rules as the parser.** Two classes of user-role lines were reported as `user_prompt`: Claude harness wrappers (slash-command markers, `` echoes, task notifications), injected meta messages (skill bodies), and compaction carriers — 233 "prompts" today of which 79 were typed by a human; and Codex 0.147 raw `response_item` messages, including the injected AGENTS.md envelope, while the real `item_completed` UserMessage/AgentMessage rows rendered as bare `item_completed` with no text. `sessionlog` now reuses `parser.ClassifyUserText` and the Codex TurnItem decoder (`DecodeCompletedTurnMessage`, exported), demotes legacy/raw duplicates in a 0.147 rollout to `legacy_message`/`model_input`/`model_output` (records kept, counts fixed), and re-tallies session kinds and preview after demotion. `user_prompts`/`assistant_messages` metrics and `insight` reports built on them count the conversation once. +- **`ccx log` previews Claude tool results.** A `tool_result` block's payload lives under `content`; the preview only looked at `text`/`message`, so every Claude tool result row read as the literal word `tool_result`. +- **Heredoc bodies no longer count as shell redirects.** `extractRedirectPaths` scanned the whole Bash command, so a Go `if n > 0` or a markdown `> 2026-08-18` inside `python3 - <<'EOF'` / `cat > f <<'EOF'` became "edited files" (`.../0`, `.../2026-08-18`) in `trace` `files_edited` and in session connections. Heredoc bodies are stripped before the redirect scan. +- **`-n` is the `--limit` shorthand everywhere.** Only `search` had it; `sessions -n 2` failed with "unknown shorthand flag". `sessions`, `projects`, and `log` now accept `-n` too. +- **A session whose summary matched dropped its content evidence.** The summary hit short-circuited the content scan, so under `--content` the session where the term was actually discussed could be the one result with no hit count, previews, or first-hit time. Summary hits stay typed `session` but now carry the content fields. +- **Codex 0.147 conversations render again.** Codex moved its UI-facing user and assistant records from legacy `event_msg.user_message` / `agent_message` events to canonical `event_msg.item_completed` TurnItems. The Codex adapter now selects one conversation source per rollout, reads stable `UserMessage` / `AgentMessage` item IDs and content, keeps legacy rollouts working, and never mistakes raw `response_item` model input (which can contain injected instruction envelopes) for a human prompt. Discovery metadata, terminal view, web, export, search, and trace now agree; the parse-cache format is bumped so upgrades cannot serve blank cached sessions. +- **`search --hits` citations print the whole message id.** The MESSAGE column cut every id to 8 characters. That is unambiguous for a uuid and wrong for the synthetic ids: `codex-thinking-90` and `codex-thinking-331` both printed as `codex-th`, and a prompt-history anchor at line 4426 printed as `line:442` — a line that exists, and is not the one cited. The citation was not merely unpasteable into `ccx view --at`; it pointed elsewhere. Only hex-prefixed ids are shortened now. + ## [0.15.0] - 2026-08-11 ### Added diff --git a/README.md b/README.md index 328958e..600b862 100644 --- a/README.md +++ b/README.md @@ -75,11 +75,15 @@ ccx sessions --provider=cx # Codex sessions in this workspace ccx sessions --after=2026-03-01 # Date filtered ccx sessions --scope yesterday --tz +8 --all --json # Session containers by end time ccx view [session] # View in terminal +ccx view [session] --at MSG_ID # Around one cited message (search --hits / trace message_id) ccx export --shape brief # Export conversation-only HTML ccx export --shape human # Only the human's turns, citable ccx trace [session] -o trace.json # Extract evidence for context folding +ccx related [session] # Which sessions connect to this one (fork, handoff, mentions, shared files) ccx log --scope yesterday --tz +8 --all --json # Time-sliced log evidence ccx search "auth bug" # Search across sessions + memory +ccx search --content -w --sort first goose # Whole-word content hits, earliest first +ccx search --content -w --hits goose # Every mention, quoted + anchored (time, session, message id) ccx run ccx-recap --agent claude # Run a bundled skill via an agent CLI ccx fork abc123 # Fork session to current project ccx doctor # Check setup diff --git a/docs/design/0004-transcript-adapter-contract.md b/docs/design/0004-transcript-adapter-contract.md new file mode 100644 index 0000000..1843b4b --- /dev/null +++ b/docs/design/0004-transcript-adapter-contract.md @@ -0,0 +1,115 @@ +# Transcript adapter contract + +ccx currently supports Claude Code, Codex, and Grok. The next target set +also includes cctrace, Gemini CLI, Kimi Code, opencode, pi, dsh, and +Cursor. That is not a request for seven more ad hoc parsers. Some sources +are conversations, some may be lineage/orchestration records, and some +may be derived evidence. Each source needs a discovery spike before ccx +classifies it. + +The Codex 0.147 break exposed the architectural rule: provider wire +formats churn; renderers must depend on a stable ccx transcript contract, +and format drift must be visible rather than becoming an empty page. + +## Layers + +```text +provider homes / indexes / files + | + v +provider-native reader + versioned wire adapter + | + v +ccx Transcript + Lineage + Diagnostics + | + +--> legacy Message-tree projection (migration only) + +--> turn/step analysis + +--> terminal, web, export, search +``` + +Provider-native structs stay inside `internal/provider/`. Do not add +one cross-provider union of every upstream event type. Shared code begins +after native records have been reduced to ccx semantics. + +## Stable domain + +The provider-neutral transcript needs these concepts explicitly: + +- `SessionRef`: identity, provider, source path, time bounds, cwd, model, + source format/version, archive state. +- `Transcript`: ordered turns/blocks plus a declared topology. +- `Topology`: linear turns, message tree, or linked child sessions. Do + not force Claude branches and Codex child threads into one tree. +- `Block`: human text, assistant text, reasoning summary, tool call, + tool result, compaction, system/meta, attachment, unknown. +- `Lineage`: parent/child session edges separate from transcript nesting. +- `Capabilities`: whether tokens, cost, errors, files, reasoning, + branching, and resume metadata are actually reported. Missing is not + zero. +- `Diagnostics`: malformed records, unknown variants, unsupported format, + dropped records, and lossy projections. Diagnostics are findings and + must reach CLI/web consumers. + +`parser.Session` / `parser.Message` is currently the renderer contract and +is still Claude-shaped. Keep it as a compatibility projection while the +neutral domain lands; do not make it the native model for the next +provider. + +## Adapter rules + +1. Select an authoritative upstream source for each semantic class. + Model request/response history, UI events, hooks, and tool lifecycle + records are not interchangeable. +2. Record the observed source format and producer version. Prefer a real + format discriminant; otherwise use explicit feature detection. +3. Unknown variants survive as diagnostics or `unknown` blocks. Never + silently turn a non-empty source into an empty transcript. +4. Deduplicate by upstream stable ID. Timestamp/text heuristics are a + documented legacy fallback only. +5. Preserve raw source anchors without exposing raw private envelopes in + rendered conversation text. +6. Keep discovery quick parse and full parse semantically identical. +7. Do not infer cost, error state, or token semantics when the provider + does not report them. +8. Provider homes remain read-only. + +## Provider acceptance gate + +Support is not complete until one sanitized fixture set and one shared +contract suite prove all applicable surfaces: + +- format reference names producer version and observed artifacts; +- fixtures cover plain chat, tools, compaction/resume, and lineage where + the provider has them; +- list metadata equals full-parse metadata; +- terminal view, web, export, search, and trace use the same transcript; +- unknown format/variant tests produce visible diagnostics; +- raw instruction, auth, and environment envelopes do not render as + human turns; +- no unsupported metric is displayed as measured zero; +- a read-only live smoke passes before release. + +The schema audit command must dispatch through the same provider adapter +registry. A separate hand-maintained Claude field list cannot enforce this +gate. + +## Sequence + +1. Land the Codex 0.147 adapter fix and regression fixture. +2. Introduce `Transcript`, `Topology`, `Capabilities`, and `Diagnostics`, + plus a projection back to the current message tree. +3. Move Claude Code, Codex, and Grok behind the contract without changing + their rendered output; add the shared acceptance suite. +4. Replace the Claude-only schema audit with provider-dispatched audits. +5. Spike each remaining target against real local artifacts, write its + format reference, sanitize fixtures, then implement one adapter at a + time. Do not promise verbs before the fixture proves the data exists. +6. Model dsh/cctrace orchestration or derived evidence as lineage/source + layers if their artifacts are not native conversations; do not fake + them into chat messages. + +Target CLI versions observed for this planning snapshot (2026-08-17) are +inputs to the spikes, not compatibility promises: Claude Code 2.1.234, +cctrace 0.40.0, Codex 0.147.0, Gemini CLI 0.55.1, Grok CLI 1.0.5, +Kimi Code 0.36.1, opencode 1.18.18, pi 0.84.2, dsh 0.1.0-rc.7, and +Cursor 2026.08.11-e8db854. diff --git a/docs/design/0005-evidence-citations-lessons-from-semantica.md b/docs/design/0005-evidence-citations-lessons-from-semantica.md new file mode 100644 index 0000000..8eef044 --- /dev/null +++ b/docs/design/0005-evidence-citations-lessons-from-semantica.md @@ -0,0 +1,76 @@ +# Design: Evidence citations — what ccx takes from Semantica + +**Created**: 2026-08-18 +**Status**: Accepted (principles + shipped items); Proposed (roadmap) +**CLI**: `ccx search --hits`, `ccx search -w / --sort / FIRST` +**Source studied**: `reference/semantica-agi/semantica` @ 5c2901a (cloned +2026-08-18, 8.7k stars, "Graph-Native Infrastructure for Context and +Accountable AI Systems") and the org profile README. + +## Why we looked + +Semantica and ccx answer the same question for different subjects: +*what was decided, why, and what evidence supports it*. Semantica does +it for enterprise AI decisions (a Python graph library: context graph, +decision records, W3C PROV-O provenance, causal chains, temporal +queries). ccx does it for coding-agent sessions (a Go CLI over the +transcripts Claude Code / Codex / Grok leave behind). The overlap is +the methodology, not the code. This note records which of their +principles we adopt, how each maps onto ccx, and what we reject. + +## Principles adopted + +| # | Semantica principle (where) | ccx mapping | Status | +|---|---|---|---| +| 1 | Explain the observable trail, not the model's cognition. "Semantica explains what's *outside* the model: the context fed in, the decision produced, its provenance, the execution trail" (`docs/concepts.md:20`). | Already ccx's stance: `trace` is "a factual record with zero interpretation; judgment belongs to the skills" (`internal/trace/types.go:10`). We keep the line hard: CLI = facts, skills = judgment. | held | +| 2 | Every fact carries a **verbatim quote plus a location** (`ProvenanceEntry.source_quote`, `source_location`, `semantica/provenance/schemas.py`). A decision without a quotable span is an assertion, not a finding. | Search results were counts per session. `--hits` makes each match a citation: time, session, role, message id, quote (`ccx search --content -w --hits X`). `trace` already anchors steps to `message_id`/`tool_id`. Rule: any ccx output that names a fact carries (session, message id, time). | shipped | +| 3 | Decision = **scenario / reasoning / outcome** as separate fields, plus who decided (`decision_models.py:87`). | Trace already carries the triple implicitly: `Turn.user_text` (scenario), `Step.narration` (reasoning at the moment of decision), `Step.mutations` + linked commits (outcome), `session.model` (decision maker). Make the mapping explicit in the recap/retro skill prompts; no schema change. | doc | +| 4 | **Supersedes ≠ derives-from** (`previous_version_id` vs `derived_from_id`); retraction is a tombstone, not a delete (`prov:Invalidation`). | `Turn.superseded` / `superseded_by_turn` already keeps the edited-away prompt as evidence. Missing: cross-session lineage (fork/resume) as a derives-from edge. | partial | +| 5 | **Bi-temporal**: when a fact was true vs when it was recorded (`kg/temporal_model.py`). | `search` now reports `FIRST` (when a term entered the record) beside `LAST` (session activity); `--sort first` answers "when did we first say X". `log --scope` slices by record time. | shipped | +| 6 | Truncation is explicit, never silent (`trace_decision_causality` appends `{"truncated": true, ...}`). | Already a ccx rule ("showing N of M (raise with -n)"); `--hits` follows it. | held | +| 7 | Discrete strength bands + one-sentence interpretation from **one shared threshold function** (`utils/helpers.py:586`), not invented decimals. | For skills: state claim strength as verified / claimed / inferred (ccx-recap already does); never emit a confidence float the data cannot support. | held | +| 8 | Symptom-named regression tests as readability contracts (`test_add_decision_scenario_stored_as_content`: humans got an opaque id where prose was expected). | Adopt the naming: e.g. `TestSessionSearcherSummaryHitKeepsContent` (this session) names the symptom, not the function. | adopted | + +## Rejected (for ccx) + +- Graph database / embeddings / vector precedent search. ccx is a + single static binary over files; lexical search with word + boundaries answers "was X ever discussed" exactly, and skills do + the semantic step. +- PROV-O / RDF export, policy engine, SHACL. Compliance theatre for our + use; the transcript file *is* the primary evidence and ccx is + read-only over it. +- Hash-chained records. Tamper evidence matters when the store is + yours; ccx does not own the store. +- God objects and alias APIs (three public names per behavior). Keep + one name per behavior; name by the user's question. +- Uncalibrated confidence floats multiplied through "decay". Ordering + is useful; the decimals are not. + +## Shipped in this pass (2026-08-18) + +- `search -w/--word`; `FIRST` column / `first_hit` / `--sort hits|first|last`; + `--hits` citations; summary hits keep content evidence; parallel scan + (~10x) with progress; `-n` shorthand on `sessions`/`projects`/`log`. + Devlog: `docs/devlog/2026-08-18-search-word-boundary-dogfood.org`. + +## Roadmap (small, ordered) — status 2026-08-18 + +1. DONE `ccx log --match QUERY [-w]` + `--kind`: records containing a + term inside a time window — the bi-temporal slice (`search --hits` + is unbounded in time). Commit afa3785. +2. DONE `ccx view --at [--context N]`: walk from + a citation to its context. Commit 8d8f77a. +3. DONE prompt history (`~/.claude/history.jsonl`, `~/.codex/history.jsonl`) + as a `type: prompt` search source, deduplicated against live + sessions. Commit bb9f07f. +4. DONE session connections: `ccx related` and `related` in + `trace --full` (docs/design/0006-session-connections.md). Commit + 8819774. Human interventions (interrupts/denials) as first-class + trace/log facts: 3a8644f. +5. DONE `trace` labels narration-less steps ("(no narration) Bash x3, + Read"); a step never renders as a bare badge row. 3a8644f. + +Next candidates: web session page shows `related` and intervention +badges (needs a visual pass); Codex approval rejections as denials; +`sessions --json` carrying interrupts/denials from quick parse. diff --git a/docs/design/0006-session-connections.md b/docs/design/0006-session-connections.md new file mode 100644 index 0000000..d82b869 --- /dev/null +++ b/docs/design/0006-session-connections.md @@ -0,0 +1,74 @@ +# Design: Session connections + +**Created**: 2026-08-18 +**Status**: Accepted +**CLI**: `ccx related [session]`; `related` in `ccx trace --full` +**Follows**: docs/design/0005-evidence-citations-lessons-from-semantica.md +(principle 2: every claim carries a quote + location; principle 4: +supersedes vs derives-from) + +## Problem + +ccx treats sessions as islands. The evidence for what happened across a +piece of work — the handoff a later session picked up, the fork that +carried a conversation into a new session, the second agent working the +same files at the same time, the session that said "see 736a7bac" — is +all in the transcripts, but nothing joins it. Every "how did we get +here" question that spans sessions falls back to eyeballing timestamps. + +The MineContext framing (capture → process → consume) puts this in the +process layer: capture is free (the JSONL is already there); the missing +processing is *connect*. Semantica's framing: a decision chain that +cannot cross a session boundary is truncated at exactly the point where +context was lost. + +## What a connection is + +A connection is a deterministic relation between two sessions, backed +by evidence a reader can walk to (message id, time, path, quote). No +LLM, no similarity scores. Relations are stated from the anchor +session's point of view: + +| Relation | Signal | Evidence carried | Strength | +|---|---|---|---| +| `forked_from` / `fork_of` | the two transcripts share message UUIDs (Claude Code fork and `ccx fork` copy history verbatim); the earlier session is the origin | shared count, first shared uuid | strong | +| `mentions` / `mentioned_by` | conversation text contains ≥8 hex chars matching the other session's id prefix | message id, time, quote | strong | +| `handoff_from` / `handoff_to` | a baton file (`HANDOFF*.md`, `*/handoffs/*`, `*devlog*`, `PLAN.md`, `TODO.md`) written by one session and read by the other later | path, writer msg id + time, reader msg id + time | strong | +| `builds_on` / `built_on_by` | a workspace file edited by one session and read or edited by the other later | up to 5 paths with both anchors, total count | medium | +| `overlaps` | time windows intersect (concurrent agents) | overlap window | medium | +| `previous` / `next` | nearest earlier / later session in the same project | start time | weak | + +Strength is a band, never a decimal (0005 principle 7). A pair can carry +several relations; the pair's strength is the strongest one. + +Scope: sessions in the same project (workspace) across providers. +`--all` widens id resolution like `trace --all`; relation search stays +per project because file-based signals only mean something inside one +workspace. + +## Surface + +``` +ccx related # latest workspace session +ccx related 736a7bac # by id / prefix / @N +ccx related --json # full evidence +ccx trace --full # bundle gains "related" +``` + +Text output: one row per related session, strongest first, then by +time: STRENGTH, SESSION, RELATIONS, WHEN, EVIDENCE (one bounded line). +Silent-cap rule holds: `-n` limits rows with "showing N of M". + +## Cost + +Every session in the project is parsed once (parse cache makes repeats +cheap); scan runs on the same bounded worker pool as `search`. Progress +on stderr when it is a terminal. + +## Non-goals + +- Semantic similarity between sessions (skills' job). +- Cross-workspace file links (paths are only comparable inside one + workspace). +- Persisting a graph. The relations are recomputed from the transcripts; + the transcripts are the store. diff --git a/docs/devlog/2026-08-17-codex-0147-rollout-drift.org b/docs/devlog/2026-08-17-codex-0147-rollout-drift.org new file mode 100644 index 0000000..f3190a1 --- /dev/null +++ b/docs/devlog/2026-08-17-codex-0147-rollout-drift.org @@ -0,0 +1,68 @@ +* [2026-08-17] Dev Log: Codex 0.147 rollout drift :CODEX:RELIABILITY: + +** Context +Codex 0.147 sessions appeared in ccx's lists but had =(no summary)=, +zero messages, zero turns, and a blank web conversation. Tool calls +still appeared, which made the failure look like a renderer bug. + +** Evidence +- The rollout still uses =session_meta=, =turn_context=, =event_msg=, + and =response_item= top-level records. +- The legacy =event_msg.user_message= and =agent_message= variants are + absent from a native 0.147 rollout. +- Visible conversation records are now =event_msg.item_completed= + carrying =TurnItem::UserMessage= and =TurnItem::AgentMessage=. +- Those items have stable IDs and typed content arrays. User text uses + =type: text=; agent text currently uses =type: Text=. +- =response_item.message= is not the visible transcript contract. Its + user-role records can include injected instruction and environment + envelopes. Rendering those would expose harness context as if the + human typed it. + +This was a parser/source-selection regression, not an HTML problem. + +** Fix +- Added a Codex-native TurnItem wire adapter in + =internal/provider/codex/turn_items.go=. +- Full parse detects whether a rollout contains completed conversation + TurnItems, then selects exactly one message source for the whole + parse: completed TurnItems when present, legacy message events + otherwise. This prevents hybrid/migrated rollouts from rendering both. +- Quick parse counts legacy and completed messages separately in one + scan, then selects the completed totals and summary when present. +- Raw =response_item.message= records remain deliberately ignored; + response-item tool calls remain supported. +- Completed item IDs become ccx message UUIDs instead of synthetic line + IDs, preserving stable anchors for export and drill-down. +- Parse cache format bumped from 3 to 4. + +** Tests +- Sanitized 0.147 fixture-in-code covers canonical item selection, + legacy/hybrid deduplication, summary and count parity, stable IDs, + model metadata, and instruction-envelope exclusion. +- Existing Codex provider tests cover the legacy fallback. +- Live read-only smoke covered =sessions=, =view --brief=, =trace=, + markdown export, the global sessions API, and the web session route. + +** Dogfood findings +1. *Schema drift fails silently.* An unsupported conversation variant + degrades to =(no summary)= and zero messages with no parser warning. + This fix restores 0.147 but does not make the next drift loud. + Fix direction: provider parse diagnostics on =Session=, surfaced in + CLI/web, with unknown top-level/payload/item variant counts. +2. *=ccx-audit-schema= is Claude-only.* The command audits Claude's + =rawMessage= fields and cannot audit Codex or Grok despite the + multi-provider product contract. Fix direction: a provider-dispatched + audit interface sharing each adapter's known wire variants. +3. *=trace= can emit an unlabeled tool-only step.* The outline showed a + numbered step containing only a tool-count badge. Fix direction: + derive a bounded label from the tool sequence or attach it to the + adjacent narration step. + +The first two findings belong to the transcript-adapter work, not as +more conditionals in the Codex parser. See +=docs/design/0004-transcript-adapter-contract.md=. + +** Privacy +No real session ID, transcript text, home path, raw rollout, or cost is +committed. The regression fixture is synthetic and sanitized. diff --git a/docs/devlog/2026-08-18-search-word-boundary-dogfood.org b/docs/devlog/2026-08-18-search-word-boundary-dogfood.org new file mode 100644 index 0000000..8dcf0e0 --- /dev/null +++ b/docs/devlog/2026-08-18-search-word-boundary-dogfood.org @@ -0,0 +1,92 @@ +* [2026-08-18] Dev Log: Dogfood findings — "when did we mention X" via search :SEARCH: + +** Context +Question: "when did we first mention semantica / semantica-agi?" Answered +against the real host store (~/.claude 4411 jsonl / 3.5G, ~/.codex 1026, +~/.grok) with a from-source build (v0.15.0-dirty, main @ d3e00c9 + WIP +codex rollout drift). Answer: never before this session — every one of +ccx's 46 "semantica" content hits was the word "semantic*ally*". The +answer was findable, but only by cross-checking with =/usr/bin/grep -w=; +ccx alone could not distinguish 0 real hits from 46. Findings recorded, +not fixed. + +** Findings +1. Substring-only matching, no word-boundary or regex option. + =search --content semantica= returned 47 results, 46 of them + "semantically". There is no =-w= / =--regex= / =--exact= flag and + the phrase syntax gives no way to express a boundary ("semantica " + misses "semantica." and "semantica-agi"). For a project/product + name that is a prefix of a common English word, the tool cannot + answer "was this term ever used"; grep -w can. Repro: + =ccx search --content semantica -n 0 --json | jq length= -> 47; + =grep -rlwi semantica ~/.claude/projects= -> 1 (this session). +2. No way to sort or filter by first-mention time. Results rank by + Priority then hit count; LAST is session end. A "when did we first + say X" question needs the timestamp of the earliest matching + message and an oldest-first order. Neither exists; the top result + is whichever session repeated the term most. +3. Cold content scan is slow and silent: 6m18s wall (226s user, 168s + sys, 104% cpu) for one query on the bind-mounted store; warm re-run + 1m14s; grep -rli on the same tree ~48s wall / 5s user. Causes are + visible in internal/cmd/search.go: (a) the loop over sessions is + sequential — zero goroutines — so a 10-core box uses one; (b) + =countMatchingLines= allocates =strings.ToLower(line)= for every + line of every file (3.5G of allocations); (c) the prefiltered + candidates then get a full =ParseSession=. Nothing is printed until + the whole scan finishes, so a 6-minute run looks hung (the harness + backgrounded it at 300s). At minimum: a stderr progress line + ("scanned 1200/4411 sessions, 12 candidates") and results streamed + as found; ideally a worker pool + a non-allocating case-fold match. +4. Content search only covers session files; =~/.claude/history.jsonl= + (every prompt since 2025-09-28, older than any surviving session + file at 2025-12-07) is not searched. For "when did we mention X" + questions the prompt history is the longest-lived evidence and the + cheapest to scan (8 MB); today it needs raw grep. +5. Small: the current session appears typed =session= (summary match, + Priority 1) and is therefore never scanned for content, so its + content hits and previews are missing from =--json= even though it + is the only real match. When a session matches both summary and + content, the content evidence should still be attached. + +** Fix directions +- =--word= (or make it the default for single-token queries with an + escape hatch) and =--regex=; both compose with =--content= / =--raw=. + Prefilter stays substring; boundary/regex applies at count time. +- Per-result =first_hit= timestamp (message time of the earliest + match) plus =--sort first|last|hits= and =--oldest=; print FIRST + alongside LAST when =--content= is on. +- Parallel scan across sessions (bounded worker pool), case-fold + without per-line allocation, progress on stderr, stream results. +- Include =history.jsonl= as a searchable source (=type: prompt=, + carries project + timestamp). +- Attach content previews to session-summary matches when =--content= + is on. + +** Resolution [2026-08-18] +Findings 1, 2, 3, 5 fixed in the same session (internal/cmd/search.go); +finding 4 (history.jsonl as a source) stays open. +- =FEAT= =-w/--word=: textMatcher with ASCII =\b= on the word-char sides + of the query, applied to names, summaries, conversation text, and + =--raw= lines. Live: =search --content -w semantica= -> 1 result + (this session) vs 47 without =-w=; agrees with =grep -rlwi=. +- =FEAT= first-hit time per content result: =FIRST= column, =first_hit= + (RFC3339) in =--json=, =--sort hits|first|last=. Verified: first_hit + 2026-08-18T07:48:10Z == earliest raw line with the word. +- =PERF= bounded worker pool (min(NumCPU, 8)), non-allocating + case-fold line match, prefilter stops at first hit, stderr progress + when a TTY. Warm scan 1m14s -> 7.6s on the same 3.5 GB store. +- =FIX= summary hits keep content evidence (matches, previews, + first_hit) instead of short-circuiting the scan. +- Tests: TestTextMatcherWord, TestIndexFold, TestLineTimestamp, + TestSessionSearcherSummaryHitKeepsContent, TestSessionSearcherMatchAllOrder, + TestSortSearchResults; existing scan tests updated to the new + signatures. +- =FEAT= (after studying semantica-agi/semantica; see + docs/design/0005-evidence-citations-lessons-from-semantica.md) + =--hits=: every matching message as a citation — time, session, + role, message id, quote — oldest first; =--raw --hits= per line. + Live: =search --content -w --hits semantica= -> 11 rows, first + c8bd2144 @ 07:48:10Z, matching the raw file. Test: + TestSessionSearcherHits. +- =FIX= =-n= shorthand for =--limit= on sessions/projects/log + (=sessions -n 2= failed "unknown shorthand flag"). diff --git a/docs/devlog/2026-08-18-session-connections-human-loop.org b/docs/devlog/2026-08-18-session-connections-human-loop.org new file mode 100644 index 0000000..c98c94d --- /dev/null +++ b/docs/devlog/2026-08-18-session-connections-human-loop.org @@ -0,0 +1,90 @@ +* [2026-08-18] Dev Log: Session connections, human interventions, citations walk :TRACE:LOG:SEARCH:RELATED: + +** Context +Eric's brief: "continue to think and improve ccx; inspired by +MineContext" with the core values spelled out — session trajectory, +how decisions were made, traceable tool calls, long context folded +short, the whole timeline and the humans in the loop, all agent +context, and the connections between sessions. Autonomous run, +commit-and-continue. Two references studied: semantica-agi/semantica +(design 0005: every fact carries a quote + location; supersedes vs +derives-from; bands not decimals) and volcengine/MineContext (capture +-> process -> consume; for ccx capture is free, the missing layer was +*connect*). + +** Why +Against the value list, the store answered "what happened in this +session" well and "how sessions relate", "where did the human step +in", and "walk me to that citation" badly or not at all: +- sessions were islands: forks, handoffs, parallel agents, and + "see 736a7bac" mentions were in the transcripts, uncomputed; +- "[Request interrupted by user]" parsed as a prompt and opened a fake + turn; permission rejections counted as tool errors; Codex + turn_aborted was invisible — 650 + 145 + 144 human interventions in + one real store, none named; +- =ccx log= reported 233 "prompts" for one day, 79 typed by a human, + and rendered Codex 0.147 conversations as bare =item_completed=; +- search/trace handed out message ids nothing could open; +- prompts older than the oldest session file were unreachable. + +** What +- =FEAT= =ccx related [session]= + =related= in =trace --full= + + =GET /api/related//= (design 0006): forked_from / + fork_of (shared message uuids), mentions / mentioned_by (id prefix in + conversation text, quoted), handoff_from / handoff_to (baton file + written by one, read by the other later), builds_on / built_on_by, + overlaps, previous / next. Strength bands; capped path lists with + count + truncated; evidence = session, message id, time, path, + quote. =trace.RelateWorkspace= shared by CLI and web. +- =FEAT= human interventions: parser =KindInterrupt= (Claude + "[Request interrupted by user…]", Codex =turn_aborted(interrupted)=) + and =IsToolDenial=; trace counts interrupts/denials on turn, step, + stats; denied calls marked =denied= (not errors); outline header and + badges; log kinds =interrupt= / =tool_denied=. +- =FEAT= =ccx log --kind K1,K2= and =--match PHRASE [-w]= — the firehose + becomes a timeline: "the humans in the loop, today" is + =ccx log --scope today --all --kind user_prompt=. +- =FEAT= =ccx view --at [--context N]= — walk from + a citation to its context (open since 08-03 finding 4). +- =FEAT= prompt history (=~/.claude/history.jsonl=, =~/.codex/history.jsonl=) + as a =search --content= source, type =prompt=, only for prompts whose + session file is gone. +- =FIX= =ccx log= applies the parser's rules: Claude command markers / + echoes / notifications / meta / compaction are not prompts; Codex + 0.147 item_completed rows are the conversation, raw response_item + and duplicated legacy events demoted (=model_input= / + =model_output= / =legacy_message=); metrics count once; tool_result + rows preview content. +- =FIX= heredoc bodies were scanned as shell redirects (=if n > 0= + inside =python3 - <<'EOF'= became an "edited file"). +- =CHANGE= narration-less trace steps get "(no narration) Bash x3, + Read" instead of a bare badge row (08-17 finding 3). + +** How +Commits, in order: 8819774 related · afa3785 log kinds + filters · +3a8644f interventions + step labels · 8d8f77a view --at · bb9f07f +history source · c0d07a5 skills/roadmap · 0f79cba /api/related + +RelateWorkspace · a1307a3 codex interrupts. Each with symptom-named +tests; =go test ./...= green throughout. + +** Decisions +| Decision | Alternatives | Rationale | DRI | Timestamp | +| relations recomputed from transcripts, no graph store | SQLite edge table | the transcripts are the store; ccx is read-only over them (0005 rejected list) | Fable 5 | 2026-08-18 | +| strength = bands, never floats | similarity scores | 0005 principle 7; a fork or a handoff is strong by kind, not by arithmetic | Fable 5 | 2026-08-18 | +| history prompts only when the session is gone | always list | one prompt, one citation; the session file is the better anchor while it exists | Fable 5 | 2026-08-18 | +| interrupt is a Kind, not a flag on user_prompt | flag | it must never anchor a turn; renderers' default branches handle it | Fable 5 | 2026-08-18 | +| /api/related endpoint only, no web panel yet | panel now | no display here to verify a UI change against the 0002 shell contract | Fable 5 | 2026-08-18 | + +** Dogfood findings (open) +1. =--match -w X= is a footgun: cobra takes =-w= as the value of + =--match=. Documented as =--match X -w=; a =--match= that rejects + values starting with =-= would be kinder. +2. Web session page does not yet show =related= or interrupt/denial + badges; the API and trace fields are there. Needs a visual pass. +3. Codex approval rejections (the counterpart of Claude's denial) were + not found as a distinct event in the store; only interrupts are + covered for Codex. +4. =related= scans every session of the workspace; a workspace with + thousands of sessions will feel it on a cold cache (progress shows, + parse cache makes the second run ~1s). A time window flag would + bound it if that bites. diff --git a/docs/schema.md b/docs/schema.md index 1703fc3..7dcfd93 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -14,8 +14,9 @@ they do not know. |------------------|------------------------------|----------| | `ccx.outline.v1` | `ccx trace --json` | Session skeleton: every turn and step headline with rollups. Read this first; it always fits. | | `ccx.turn.v1` | `ccx trace --turn N` | One turn with full step evidence, plus the sidechain entries that turn references, plus warnings. | -| `ccx.trace.v2` | `ccx trace --full` | Complete evidence bundle: all turns/steps, sidechains, git correlation, workspace context, stats, warnings. Large. | -| `ccx.log.v1` | `ccx log --json`, `ccx insight --json` | Time-scoped records across sessions with pre-computed `days[]` / `providers[]` / `workspaces[]` aggregates. | +| `ccx.trace.v2` | `ccx trace --full` | Complete evidence bundle: all turns/steps, sidechains, git correlation, workspace context, `related` sessions, stats, warnings. Large. | +| `ccx.related.v1` | `ccx related --json`, `GET /api/related//` | The anchor session's connections to the other sessions of its workspace: `related[]` of `{session_id, provider, summary, start, end, strength, relations[]}`, plus `total`/`shown`. Each relation is `{kind, count?, paths?, evidence[], truncated?}`; evidence items are `{session_id, message_id, time, path?, quote?}`. Kinds: `forked_from`/`fork_of`, `mentions`/`mentioned_by`, `handoff_from`/`handoff_to`, `builds_on`/`built_on_by`, `overlaps`, `previous`/`next`. Strength is `strong`/`medium`/`weak`. | +| `ccx.log.v1` | `ccx log --json`, `ccx insight --json` | Time-scoped records across sessions with pre-computed `days[]` / `providers[]` / `workspaces[]` aggregates. Record `kind` is provider-normalized: `user_prompt` and `assistant_message` are the visible conversation only (Claude command markers/echoes/notifications are `command`/`command_output`/`notification`, injected meta is `meta`, compaction carriers `compact_summary`; Codex 0.147 raw `response_item` messages are `model_input`/`model_output` and duplicated legacy events `legacy_message`). With `--kind`/`--match`, `metrics.records` stays scope-wide and `metrics.records_matched` reports the narrowed count before any limit. | ## Versioning policy @@ -49,6 +50,14 @@ gaps, each capped at 5 minutes. Continuous work counts fully; an overnight gap counts as at most one cap. Use active time for "how long did this actually take"; use wall-span for calendar placement. +### Human interventions + +`interrupts` (the "[Request interrupted by user]" marker: the human +pressed stop) and `denials` (a tool call rejected at the permission +prompt) are counted on turns, steps, and `stats`; a denied call carries +`denied: true` on its evidence and is not counted as an error (it did +not run). Neither is a prompt: an interrupt never opens a turn. + ### Facts, not judgment Traces record what happened: text excerpts, tool calls, mutations, diff --git a/internal/cmd/log.go b/internal/cmd/log.go index 5d5e0e3..357ccaf 100644 --- a/internal/cmd/log.go +++ b/internal/cmd/log.go @@ -24,11 +24,19 @@ Sessions can run for days or months, so time-scoped review must use log records, not just session end times. This command emits the evidence layer for scoped insight: records inside the window plus session overlap metadata. +--kind narrows to record kinds (comma-separated: user_prompt, +assistant_message, tool_call, tool_result, reasoning, ...); --match +keeps records whose raw transcript line contains a phrase (-w for +whole words). Together they turn the firehose into a timeline: +every human prompt today, across every session, in order. + Examples: ccx log --scope yesterday --tz +8 --all --json ccx log --since 2026-05-21 --until 2026-05-22 --tz +8 --all --json ccx log --scope week --provider cx --json - ccx log /path/to/repo --scope yesterday --json --raw`, + ccx log /path/to/repo --scope yesterday --json --raw + ccx log --scope today --all --kind user_prompt # the humans in the loop, today + ccx log --scope month --all --match semantica -w # when a term came up, in a window`, Args: cobra.MaximumNArgs(1), RunE: runLog, } @@ -43,6 +51,9 @@ var ( logAll bool logProvider string logLimit int + logKinds string + logMatch string + logWord bool logLocation *time.Location ) @@ -55,7 +66,10 @@ func init() { logCmd.Flags().BoolVar(&logRaw, "raw", false, "include raw JSONL lines in JSON output") logCmd.Flags().BoolVar(&logAll, "all", false, "slice logs across all projects") logCmd.Flags().StringVarP(&logProvider, "provider", "p", "", "filter by provider: cc, cx, all") - logCmd.Flags().IntVar(&logLimit, "limit", 0, "limit records in JSON output (0 = no limit)") + logCmd.Flags().IntVarP(&logLimit, "limit", "n", 0, "limit records in JSON output (0 = no limit)") + logCmd.Flags().StringVar(&logKinds, "kind", "", "keep only these record kinds (comma-separated, e.g. user_prompt,assistant_message)") + logCmd.Flags().StringVar(&logMatch, "match", "", "keep only records whose raw transcript line contains this phrase") + logCmd.Flags().BoolVarP(&logWord, "word", "w", false, "--match whole words only") } func runLog(cmd *cobra.Command, args []string) error { @@ -86,6 +100,16 @@ func runLog(cmd *cobra.Command, args []string) error { return fmt.Errorf("invalid --provider %q (want cc, cx, claude-code, codex, or all)", logProvider) } + var kinds []string + if strings.TrimSpace(logKinds) != "" { + kinds = strings.Split(logKinds, ",") + } + var match func(string) bool + if strings.TrimSpace(logMatch) != "" { + m := newTextMatcher(logMatch, logWord) + match = m.matches + } + settings := config.Load() bundle, err := sessionlog.Collect(logSources(settings, providerFilter), sessionlog.Options{ Start: start, @@ -99,6 +123,8 @@ func runLog(cmd *cobra.Command, args []string) error { Limit: logLimit, IncludeRaw: logRaw, Now: time.Now().In(loc), + Kinds: kinds, + Match: match, }) if err != nil { return err @@ -178,8 +204,11 @@ func printLogTable(bundle *sessionlog.Bundle) error { } fmt.Printf("ccx log · %s · %s\n", bundle.Scope.Label, bundle.Scope.TimeZone) recordsLabel := fmt.Sprintf("records %d", bundle.Metrics.Records) - if bundle.Metrics.RecordsReturned != bundle.Metrics.Records { - recordsLabel = fmt.Sprintf("records %d · showing %d", bundle.Metrics.Records, bundle.Metrics.RecordsReturned) + if bundle.Metrics.RecordsMatched > 0 || len(bundle.Records) == 0 && bundle.Metrics.Records > 0 && bundle.Metrics.RecordsReturned == 0 { + recordsLabel = fmt.Sprintf("records %d · matched %d", bundle.Metrics.Records, bundle.Metrics.RecordsMatched) + } + if bundle.Metrics.RecordsReturned != bundle.Metrics.Records && bundle.Metrics.RecordsReturned != bundle.Metrics.RecordsMatched { + recordsLabel += fmt.Sprintf(" · showing %d", bundle.Metrics.RecordsReturned) } fmt.Printf("%s to %s · source log files %d · %s\n\n", bundle.Scope.Start.Format("2006-01-02 15:04"), diff --git a/internal/cmd/projects.go b/internal/cmd/projects.go index 2f08306..87aeb45 100644 --- a/internal/cmd/projects.go +++ b/internal/cmd/projects.go @@ -29,7 +29,7 @@ var ( func init() { projectsCmd.Flags().StringVar(&projectsSort, "sort", "time", "sort by: name, time, sessions") - projectsCmd.Flags().IntVar(&projectsLimit, "limit", 0, "limit number of projects (0 = no limit)") + projectsCmd.Flags().IntVarP(&projectsLimit, "limit", "n", 0, "limit number of projects (0 = no limit)") projectsCmd.Flags().BoolVar(&projectsJSON, "json", false, "output as JSON") } diff --git a/internal/cmd/related.go b/internal/cmd/related.go new file mode 100644 index 0000000..8a8875a --- /dev/null +++ b/internal/cmd/related.go @@ -0,0 +1,237 @@ +package cmd + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "text/tabwriter" + "time" + + "github.com/spf13/cobra" + + "github.com/thevibeworks/ccx/internal/catalog" + "github.com/thevibeworks/ccx/internal/parser" + "github.com/thevibeworks/ccx/internal/provider" + "github.com/thevibeworks/ccx/internal/trace" +) + +var relatedCmd = &cobra.Command{ + Use: "related [session]", + Short: "Which sessions connect to this one, and how", + Long: `Which sessions connect to this one, and how. + +Sessions are islands in the log; the connections between them are +in the transcripts but nothing joins them. This command joins them, +deterministically, from the anchor session's point of view: + + forked_from / fork_of the transcripts share message ids + mentions / mentioned_by text names the other session's id + handoff_from / handoff_to a baton file (HANDOFF.md, handoffs/, + devlog, PLAN.md) written by one, read + by the other later + builds_on / built_on_by a workspace file edited by one, then + read or edited by the other + overlaps the two ran at the same time + previous / next nearest neighbours in the workspace + +Every relation carries evidence (message id, time, path, quote) in +--json. Strength is a band — strong, medium, weak — never a score. +Scope is the anchor session's workspace, all providers. + +Examples: + ccx related Connections of the latest workspace session + ccx related 736a7bac A specific session + ccx related --json Full evidence for scripts and skills`, + Args: cobra.MaximumNArgs(1), + RunE: runRelated, +} + +var ( + relatedProject string + relatedAll bool + relatedJSON bool + relatedLimit int +) + +func init() { + relatedCmd.Flags().StringVarP(&relatedProject, "project", "p", "", "project name") + relatedCmd.Flags().BoolVar(&relatedAll, "all", false, "resolve the session across all projects") + relatedCmd.Flags().BoolVar(&relatedJSON, "json", false, "output as JSON") + relatedCmd.Flags().IntVarP(&relatedLimit, "limit", "n", 20, "max related sessions (0 = no limit)") + rootCmd.AddCommand(relatedCmd) +} + +func runRelated(cmd *cobra.Command, args []string) error { + backend := provider.Default() + + var session *parser.Session + var err error + if len(args) == 0 { + session, err = latestTraceSession(backend, relatedAll) + if err != nil { + return fmt.Errorf("session: %w", err) + } + } else { + projectName, sessionID := parseSessionArg(args[0]) + if relatedProject != "" { + projectName = relatedProject + } + query, err := sessionLookupQuery(projectName, relatedAll) + if err != nil { + return err + } + session, err = resolveSessionInQuery(backend, query, sessionID) + if err != nil { + return err + } + } + if session == nil { + return fmt.Errorf("no session found") + } + + related, warnings, err := relateSession(backend, session) + if err != nil { + return err + } + for _, w := range warnings { + fmt.Fprintf(os.Stderr, "warning: %s\n", w.Message) + } + + total := len(related) + if relatedLimit > 0 && total > relatedLimit { + fmt.Fprintf(os.Stderr, "showing %d of %d related sessions (raise with -n)\n", relatedLimit, total) + related = related[:relatedLimit] + } + + if relatedJSON { + enc := json.NewEncoder(os.Stdout) + enc.SetIndent("", " ") + return enc.Encode(map[string]any{ + "kind": "ccx.related.v1", + "session": map[string]any{"id": session.ID, "provider": session.Provider, "project": session.ProjectName, "path": session.FilePath}, + "related": related, + "total": total, + "shown": len(related), + }) + } + return printRelated(session, related, total) +} + +// relateSession is the CLI wrapper over trace.RelateWorkspace: same +// worker pool as search, progress on stderr when it is a terminal. +func relateSession(backend provider.Backend, anchor *parser.Session) ([]trace.RelatedSession, []trace.TraceWarning, error) { + // Progress needs the total; list once, cheaply, for the count. + query := catalog.SessionQuery{Scope: catalog.ScopeProject, ProjectName: anchor.ProjectName} + if strings.TrimSpace(anchor.CWD) != "" { + query = catalog.SessionQuery{Scope: catalog.ScopeWorkspace, WorkspacePath: anchor.CWD} + } + total := 1 + if sessions, err := backend.ListSessions(query.WithoutLimit().WithoutProviderFilter()); err == nil { + total = len(sessions) + 1 + } + progress := newScanProgress(total, true) + related, warnings, err := trace.RelateWorkspace(backend, anchor, searchWorkers(true), progress.tick) + progress.done() + return related, warnings, err +} + +func printRelated(anchor *parser.Session, related []trace.RelatedSession, total int) error { + // Mirror the trace header: workspace basename beats the encoded + // project dir name. + where := anchor.ProjectName + if strings.HasPrefix(where, "-") { + where = parser.GetProjectDisplayName(where) + } + if strings.TrimSpace(anchor.CWD) != "" { + where = filepath.Base(anchor.CWD) + } + fmt.Printf("related to %s (%s): %d session(s)\n", truncateID(anchor.ID, 8), cleanDisplayText(where), total) + if len(related) == 0 { + return nil + } + w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0) + fmt.Fprintln(w, "STRENGTH\tSESSION\tPROVIDER\tRELATIONS\tSTART\tEVIDENCE") + for _, r := range related { + kinds := make([]string, 0, len(r.Relations)) + for _, rel := range r.Relations { + k := rel.Kind + if rel.Count > 1 && (rel.Kind == trace.RelBuildsOn || rel.Kind == trace.RelBuiltOnBy || rel.Kind == trace.RelHandoffFrom || rel.Kind == trace.RelHandoffTo) { + k = fmt.Sprintf("%s(%d)", k, rel.Count) + } + kinds = append(kinds, k) + } + fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", + r.Strength, truncateID(r.SessionID, 8), providerTag(r.Provider), strings.Join(kinds, ","), + formatRelatedTime(r.Start), truncateDisplay(cleanDisplayText(relationEvidenceLine(r)), 96)) + } + return w.Flush() +} + +// relationEvidenceLine renders the strongest relation's evidence as one +// bounded line; the rest is in --json. +func relationEvidenceLine(r trace.RelatedSession) string { + if len(r.Relations) == 0 { + return "" + } + rel := r.Relations[0] + for _, cand := range r.Relations[1:] { + if strengthRankOf(cand.Kind) < strengthRankOf(rel.Kind) { + rel = cand + } + } + switch rel.Kind { + case trace.RelForkedFrom, trace.RelForkOf: + id := "" + if len(rel.Evidence) > 0 { + id = truncateID(rel.Evidence[0].MessageID, 8) + } + return fmt.Sprintf("%d shared message ids (e.g. %s)", rel.Count, id) + case trace.RelMentions, trace.RelMentionedBy: + if len(rel.Evidence) > 0 { + return fmt.Sprintf("%s@%s: %s", truncateID(rel.Evidence[0].SessionID, 8), formatRelatedTime(rel.Evidence[0].Time), rel.Evidence[0].Quote) + } + case trace.RelHandoffFrom, trace.RelHandoffTo, trace.RelBuildsOn, trace.RelBuiltOnBy: + if len(rel.Evidence) >= 2 { + more := "" + if rel.Count > 1 { + more = fmt.Sprintf(" (+%d more)", rel.Count-1) + } + return fmt.Sprintf("%s%s: written %s@%s, touched %s@%s", + shortPath(rel.Evidence[0].Path), more, + truncateID(rel.Evidence[0].SessionID, 8), formatRelatedTime(rel.Evidence[0].Time), + truncateID(rel.Evidence[1].SessionID, 8), formatRelatedTime(rel.Evidence[1].Time)) + } + case trace.RelOverlaps: + if len(rel.Evidence) >= 2 { + return fmt.Sprintf("both active %s - %s", formatRelatedTime(rel.Evidence[0].Time), formatRelatedTime(rel.Evidence[1].Time)) + } + } + return "-" +} + +func strengthRankOf(kind string) int { + switch kind { + case trace.RelForkedFrom, trace.RelForkOf, trace.RelMentions, trace.RelMentionedBy, trace.RelHandoffFrom, trace.RelHandoffTo: + return 0 + case trace.RelBuildsOn, trace.RelBuiltOnBy, trace.RelOverlaps: + return 1 + } + return 2 +} + +func shortPath(path string) string { + parts := strings.Split(filepath.ToSlash(path), "/") + if len(parts) <= 3 { + return path + } + return ".../" + strings.Join(parts[len(parts)-3:], "/") +} + +func formatRelatedTime(t time.Time) string { + if t.IsZero() { + return "-" + } + return t.Local().Format("2006-01-02 15:04") +} diff --git a/internal/cmd/root.go b/internal/cmd/root.go index f5a0363..df02c7b 100644 --- a/internal/cmd/root.go +++ b/internal/cmd/root.go @@ -42,6 +42,7 @@ CLI commands: ccx view View session in terminal ccx export -f html Export to HTML/Markdown/Org ccx trace What the agent did: turn/step outline + drill-down + ccx related Which sessions connect to this one, and how ccx log Slice raw session logs by time scope ccx skills install Install bundled agent skills matching this binary diff --git a/internal/cmd/search.go b/internal/cmd/search.go index 7966a8b..934a232 100644 --- a/internal/cmd/search.go +++ b/internal/cmd/search.go @@ -7,9 +7,13 @@ import ( "io" "os" "path/filepath" + "regexp" + "runtime" "sort" "strings" + "sync" "text/tabwriter" + "time" "unicode/utf8" "github.com/spf13/cobra" @@ -26,24 +30,44 @@ var searchCmd = &cobra.Command{ The query is one case-insensitive phrase: multiple words must appear adjacent and in order ("fix bug" won't match "bug ... fix"). For -term-level matching, run one search per term. Exits 0 either way; -zero matches just prints "No results found." +term-level matching, run one search per term. Matching is substring +by default; -w/--word requires whole words, so "semantica" no longer +matches "semantically". Exits 0 either way; zero matches just prints +"No results found." With --content, also scan conversation text inside session files (including subagent files): user prompts and assistant replies, -ranked by hit count with a matched-text preview. Injected noise — -tool results, hook attachments, command echoes — doesn't count. +ranked by hit count with a matched-text preview and the time of the +earliest match (FIRST). Injected noise — tool results, hook +attachments, command echoes — doesn't count. Prompt history +(~/.claude/history.jsonl, ~/.codex/history.jsonl) is scanned too and +surfaces as type "prompt" for prompts whose session file is gone — +the longest-lived evidence, past session cleanup. Add --raw to match every raw transcript line instead: grep parity, no parse, misses nothing grep would find. +--sort orders results: hits (default: match kind, then hit count), +first (earliest match first — "when did we first mention X"), last +(most recently active session first). + +--hits lists every matching message instead of one row per session: +time, session, role, message id, and the quote around the match, +oldest first. Each row is a citation — the same anchors ccx trace +and view use — so a claim built on a search can point at its +evidence. -n caps the rows. + Examples: ccx search auth # Find sessions about authentication ccx search myproject # Find project by name ccx search "fix bug" # Phrase match: adjacent words, in order ccx search -t session # Only search sessions ccx search --content goose # Scan conversation text (slower) - ccx search --raw goose # Grep parity over raw lines`, + ccx search --raw goose # Grep parity over raw lines + ccx search --content -w --sort first semantica + # Whole-word, earliest mention first + ccx search --content -w --hits semantica + # Every mention, quoted and anchored`, Args: cobra.MinimumNArgs(1), RunE: runSearch, } @@ -58,6 +82,9 @@ var ( searchModel string searchContent bool searchRaw bool + searchWord bool + searchSort string + searchHits bool ) func init() { @@ -70,6 +97,9 @@ func init() { searchCmd.Flags().StringVar(&searchModel, "model", "", "filter by model name substring") searchCmd.Flags().BoolVar(&searchContent, "content", false, "also scan conversation text in session files (slower)") searchCmd.Flags().BoolVar(&searchRaw, "raw", false, "content scan matches every raw transcript line (grep parity; implies --content)") + searchCmd.Flags().BoolVarP(&searchWord, "word", "w", false, "match whole words only (\"semantica\" won't match \"semantically\")") + searchCmd.Flags().StringVar(&searchSort, "sort", "hits", "order results: hits, first, last") + searchCmd.Flags().BoolVar(&searchHits, "hits", false, "list every matching message with time, role, message id, and quote (implies --content)") rootCmd.AddCommand(searchCmd) } @@ -82,8 +112,13 @@ type searchResult struct { Summary string `json:"summary"` Time string `json:"time,omitempty"` Matches int `json:"matches,omitempty"` + FirstHit string `json:"first_hit,omitempty"` // RFC3339 time of the earliest match Previews []contentPreview `json:"previews,omitempty"` Priority int `json:"-"` + + firstHit time.Time + lastTime time.Time + hits []searchHit } // contentPreview is one matched conversation snippet, role-labeled so @@ -93,15 +128,151 @@ type contentPreview struct { Text string `json:"text"` } +// searchHit is one matching message: a quote plus the anchors needed +// to walk back to it (session file, message id, time). Under --raw the +// unit is a transcript line and Role is the line's top-level type. +type searchHit struct { + Project string `json:"project"` + Session string `json:"session"` + Path string `json:"path"` + MessageID string `json:"message_id,omitempty"` + Time time.Time `json:"time"` + Role string `json:"role"` + Matches int `json:"matches"` // occurrences within this message + Quote string `json:"quote"` +} + const maxContentPreviews = 3 +// textMatcher decides how QUERY matches text. Default: case-insensitive +// substring. Word mode adds ASCII word boundaries on each side of the +// query that starts/ends with a word character, so "semantica" no +// longer matches "semantically" while "中文" still matches inside a CJK +// run (no \w on either side, so no boundary is demanded there). +type textMatcher struct { + query string // lowercase literal + re *regexp.Regexp // nil in substring mode +} + +func newTextMatcher(query string, word bool) textMatcher { + m := textMatcher{query: strings.ToLower(query)} + if !word || m.query == "" { + return m + } + pat := regexp.QuoteMeta(m.query) + if isWordByte(m.query[0]) { + pat = `\b` + pat + } + if isWordByte(m.query[len(m.query)-1]) { + pat += `\b` + } + m.re = regexp.MustCompile(`(?i)` + pat) + return m +} + +func isWordByte(c byte) bool { + return c == '_' || (c >= '0' && c <= '9') || (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') +} + +// literal is the substring matcher for the same query: a superset of +// word matches, so it is a valid cheap prefilter. +func (m textMatcher) literal() textMatcher { return textMatcher{query: m.query} } + +// index returns the byte offset and length of the first match in +// text, or -1, 0. +func (m textMatcher) index(text string) (int, int) { + if m.re == nil { + return indexFold(text, m.query), len(m.query) + } + loc := m.re.FindStringIndex(text) + if loc == nil { + return -1, 0 + } + return loc[0], loc[1] - loc[0] +} + +func (m textMatcher) matches(text string) bool { + i, _ := m.index(text) + return i >= 0 +} + +// count returns the number of non-overlapping matches in text. +func (m textMatcher) count(text string) int { + if m.re == nil { + return countFold(text, m.query) + } + return len(m.re.FindAllStringIndex(text, -1)) +} + +// indexFold is a case-insensitive strings.Index for a lowercase query +// that does not allocate a lowered copy of s: the raw scan runs it +// over every transcript line, and lowering gigabytes was the scan's +// dominant cost. Non-ASCII queries fall back to the lowered copy +// (Unicode case folding is not byte-stable). +func indexFold(s, query string) int { + if query == "" { + return 0 + } + if !isASCII(query) { + return strings.Index(strings.ToLower(s), query) + } + n := len(query) + first := query[0] + for i := 0; i+n <= len(s); i++ { + c := s[i] + if c >= 'A' && c <= 'Z' { + c += 'a' - 'A' + } + if c != first { + continue + } + if strings.EqualFold(s[i:i+n], query) { + return i + } + } + return -1 +} + +func countFold(s, query string) int { + if query == "" { + return 0 + } + if !isASCII(query) { + return strings.Count(strings.ToLower(s), query) + } + n := 0 + for { + i := indexFold(s, query) + if i < 0 { + return n + } + n++ + s = s[i+len(query):] + } +} + +func isASCII(s string) bool { + for i := 0; i < len(s); i++ { + if s[i] >= utf8.RuneSelf { + return false + } + } + return true +} + func runSearch(cmd *cobra.Command, args []string) error { - query := strings.ToLower(strings.Join(args, " ")) + m := newTextMatcher(strings.Join(args, " "), searchWord) + query := m.query backend := provider.Default() - if searchRaw { + if searchRaw || searchHits { searchContent = true } + switch searchSort { + case "hits", "first", "last": + default: + return fmt.Errorf("invalid --sort %q: want hits, first, or last", searchSort) + } after, err := config.ParseDate(searchAfter) if err != nil { @@ -124,6 +295,7 @@ func runSearch(cmd *cobra.Command, args []string) error { } var results []searchResult + var candidates []sessionCandidate for _, p := range projects { // Backends set Name to the human-readable form; the encoding @@ -140,9 +312,7 @@ func runSearch(cmd *cobra.Command, args []string) error { // Project name match (skip if filtering to sessions only) if searchType != "session" { - nameMatch := strings.Contains(strings.ToLower(p.EncodedName), query) || - strings.Contains(strings.ToLower(projPath), query) || - strings.Contains(strings.ToLower(projDisplay), query) + nameMatch := m.matches(p.EncodedName) || m.matches(projPath) || m.matches(projDisplay) providerMatch := filter.Provider == "" if !providerMatch { @@ -173,100 +343,42 @@ func runSearch(cmd *cobra.Command, args []string) error { if !filter.IsEmpty() && !filter.Match(s) { continue } + candidates = append(candidates, sessionCandidate{session: s, project: projDisplay}) + } + } - // Session ID match (high priority) - if strings.HasPrefix(strings.ToLower(s.ID), query) { - results = append(results, searchResult{ - Type: "session", - Project: projDisplay, - Session: truncateID(s.ID, 8), - Path: s.FilePath, - Summary: sessionSummaryPreview(s.Summary, 64), - Time: formatAge(s.EndTime), - Priority: 0, - }) - continue - } - - // Summary match - if strings.Contains(strings.ToLower(s.Summary), query) { - results = append(results, searchResult{ - Type: "session", - Project: projDisplay, - Session: truncateID(s.ID, 8), - Path: s.FilePath, - Summary: sessionSummaryPreview(s.Summary, 64), - Time: formatAge(s.EndTime), - Priority: 2, - }) - continue - } - - // Content scan. The default counts conversation text only — - // user prompts and assistant replies — so ranking follows - // discussion, not injected boilerplate (a hook line fired - // every turn once outranked the real answer 327 hits to 13; - // docs/devlog/2026-08-03-content-search-noise.org). --raw - // keeps grep parity over raw transcript lines, main file - // plus subagent files: no parse, works for every provider's - // format, misses nothing grep would find. - if searchContent { - if searchRaw { - if n := countContentMatches(s.FilePath, query); n > 0 { - results = append(results, searchResult{ - Type: "content", - Project: projDisplay, - Session: truncateID(s.ID, 8), - Path: s.FilePath, - Summary: fmt.Sprintf("%d hits · %s", n, sessionSummaryPreview(s.Summary, 48)), - Time: formatAge(s.EndTime), - Matches: n, - Priority: 3, - }) - } - continue - } + searcher := sessionSearcher{m: m, content: searchContent, raw: searchRaw, hits: searchHits, parser: backend} + for _, r := range searcher.matchAll(candidates, searchWorkers(searchContent)) { + results = append(results, *r) + } - // Cheap line-scan prefilter before the full parse; only - // trustworthy when JSON escaping can't hide the query. - if rawPrefilterSafe(query) && countContentMatches(s.FilePath, query) == 0 { - continue - } - n, previews := scanConversationText(backend, s.FilePath, query) - if n == 0 { - continue - } - summary := fmt.Sprintf("%d hits · %s", n, sessionSummaryPreview(s.Summary, 48)) - if len(previews) > 0 { - summary = fmt.Sprintf("%d hits · [%s] %s", n, previews[0].Role, truncateDisplay(previews[0].Text, 56)) - } - results = append(results, searchResult{ - Type: "content", - Project: projDisplay, - Session: truncateID(s.ID, 8), - Path: s.FilePath, - Summary: summary, - Time: formatAge(s.EndTime), - Matches: n, - Previews: previews, - Priority: 3, - }) - } + // Prompt history: prompts whose session file is gone (cleanup) + // still exist here; the only place "when did we first say X" can + // be answered past the session horizon. + if searchContent && searchType != "project" { + settings := config.Load() + known := knownSessionIDs(candidates) + for _, h := range scanPromptHistory(promptHistoryFiles(settings.ClaudeHome, settings.CodexHome), m, known) { + results = append(results, promptResult(h)) } } + if searchHits { + return printSearchHits(collectHits(results)) + } + // Search memory files if searchType != "project" && searchType != "session" { settings := config.Load() for _, home := range []string{settings.ClaudeHome, settings.CodexHome} { - searchMemoryDir(home, "projects", query, filter, &results) + searchMemoryDir(home, "projects", m, filter, &results) // Global files for _, name := range []string{"CLAUDE.md", "instructions.md", "AGENTS.md"} { path := filepath.Join(home, name) if _, err := os.Stat(path); err != nil { continue } - if strings.Contains(strings.ToLower(name), query) { + if m.matches(name) { results = append(results, searchResult{ Type: "memory", Project: filepath.Base(home), @@ -279,14 +391,7 @@ func runSearch(cmd *cobra.Command, args []string) error { } } - // Sort by priority, then by match count within content results. - // Stable so equal-rank results keep discovery order across runs. - sort.SliceStable(results, func(i, j int) bool { - if results[i].Priority != results[j].Priority { - return results[i].Priority < results[j].Priority - } - return results[i].Matches > results[j].Matches - }) + sortSearchResults(results, searchSort) // Limit results — never silently. if searchLimit > 0 && len(results) > searchLimit { @@ -311,33 +416,98 @@ func runSearch(cmd *cobra.Command, args []string) error { return enc.Encode(results) } - return printSearchResults(results) + return printSearchResults(results, searchContent) +} + +// sortSearchResults orders results. "hits": match kind (Priority), +// then hit count — stable, so equal-rank results keep discovery order +// across runs. "first": earliest match first; results with no match +// time (projects, memory, name-only hits) trail in hits order. +// "last": most recently active session first, timeless trailing. +func sortSearchResults(results []searchResult, mode string) { + byHits := func(a, b searchResult) bool { + if a.Priority != b.Priority { + return a.Priority < b.Priority + } + return a.Matches > b.Matches + } + sort.SliceStable(results, func(i, j int) bool { + a, b := results[i], results[j] + switch mode { + case "first": + switch { + case a.firstHit.IsZero() != b.firstHit.IsZero(): + return !a.firstHit.IsZero() + case !a.firstHit.IsZero() && !a.firstHit.Equal(b.firstHit): + return a.firstHit.Before(b.firstHit) + } + case "last": + switch { + case a.lastTime.IsZero() != b.lastTime.IsZero(): + return !a.lastTime.IsZero() + case !a.lastTime.IsZero() && !a.lastTime.Equal(b.lastTime): + return a.lastTime.After(b.lastTime) + } + } + return byHits(a, b) + }) +} + +// searchWorkers sizes the content-scan pool. The scan is I/O plus +// per-line matching; one goroutine per CPU (capped) keeps a cold +// store busy without thrashing it. +func searchWorkers(content bool) int { + if !content { + return 1 + } + n := runtime.NumCPU() + if n > 8 { + n = 8 + } + if n < 1 { + n = 1 + } + return n } -func printSearchResults(results []searchResult) error { +func printSearchResults(results []searchResult, withFirst bool) error { w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0) // LAST = last activity (session end time): the same timestamp // --after/--before filter on and results sort by, so a filtered - // row never displays a date outside the requested window. - fmt.Fprintln(w, "TYPE\tPROJECT\tSESSION\tSUMMARY\tLAST") + // row never displays a date outside the requested window. FIRST = + // earliest matching message, shown when content was scanned. + if withFirst { + fmt.Fprintln(w, "TYPE\tPROJECT\tSESSION\tSUMMARY\tFIRST\tLAST") + } else { + fmt.Fprintln(w, "TYPE\tPROJECT\tSESSION\tSUMMARY\tLAST") + } for _, r := range results { session := r.Session if session == "" { session = "-" } - time := r.Time - if time == "" { - time = "-" + last := r.Time + if last == "" { + last = "-" + } + if withFirst { + first := "-" + if !r.firstHit.IsZero() { + first = r.firstHit.Local().Format("2006-01-02 15:04") + } + fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", + r.Type, truncateDisplay(cleanDisplayText(r.Project), 24), session, cleanDisplayText(r.Summary), first, last) + continue } fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", - r.Type, truncateDisplay(cleanDisplayText(r.Project), 24), session, cleanDisplayText(r.Summary), time) + r.Type, truncateDisplay(cleanDisplayText(r.Project), 24), session, cleanDisplayText(r.Summary), last) } return w.Flush() } -func searchMemoryDir(home, subdir, query string, filter config.SessionFilter, results *[]searchResult) { +func searchMemoryDir(home, subdir string, m textMatcher, filter config.SessionFilter, results *[]searchResult) { projectsDir := filepath.Join(home, subdir) projEntries, err := os.ReadDir(projectsDir) if err != nil { @@ -357,7 +527,7 @@ func searchMemoryDir(home, subdir, query string, filter config.SessionFilter, re if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".md") { continue } - if strings.Contains(strings.ToLower(entry.Name()), query) { + if m.matches(entry.Name()) { *results = append(*results, searchResult{ Type: "memory", Project: projDisplay, @@ -377,63 +547,362 @@ func truncateID(id string, max int) string { return id[:max] } +// citationID renders a message id for the MESSAGE column, which is +// the value a reader pastes into `ccx view --at`. A uuid stays +// unambiguous at its 8-hex prefix, so shorten it; a synthetic id +// (codex-thinking-13, line:442) is short already and every one of +// them collides at 8 chars, so print it whole or the citation walk +// dead-ends on "prefix is ambiguous". +func citationID(id string) string { + if len(id) > 8 && isHexRun(id[:8]) { + return id[:8] + } + return id +} + +func isHexRun(s string) bool { + for i := 0; i < len(s); i++ { + c := s[i] + if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')) { + return false + } + } + return len(s) > 0 +} + // sessionParser is the slice of provider.Backend the conversation // scan needs; narrowed so tests can stub it. type sessionParser interface { ParseSession(filePath string) (*parser.Session, error) } +// sessionCandidate is one session that passed the provider/date/model +// filters and still has to be matched against the query. +type sessionCandidate struct { + session *parser.Session + project string +} + +// sessionSearcher matches one session against the query: ID prefix, +// summary, and — with content on — conversation text or raw lines. +type sessionSearcher struct { + m textMatcher + content bool + raw bool + hits bool // collect every matching message, not just previews + parser sessionParser +} + +// matchAll runs match over candidates on `workers` goroutines and +// returns the hits in candidate order (so equal-rank results keep +// discovery order). Progress goes to stderr when it is a terminal. +func (ss sessionSearcher) matchAll(candidates []sessionCandidate, workers int) []*searchResult { + out := make([]*searchResult, len(candidates)) + if len(candidates) == 0 { + return nil + } + if workers < 1 { + workers = 1 + } + if workers > len(candidates) { + workers = len(candidates) + } + + progress := newScanProgress(len(candidates), ss.content) + var wg sync.WaitGroup + next := make(chan int) + for w := 0; w < workers; w++ { + wg.Add(1) + go func() { + defer wg.Done() + for i := range next { + out[i] = ss.match(candidates[i].session, candidates[i].project) + progress.tick() + } + }() + } + for i := range candidates { + next <- i + } + close(next) + wg.Wait() + progress.done() + + hits := out[:0] + for _, r := range out { + if r != nil { + hits = append(hits, r) + } + } + return hits +} + +// match returns the result for one session, or nil. A summary hit +// stays typed "session" but still carries content evidence (matches, +// previews, first hit) when content is on — the session where the +// term was actually discussed must not be the one result without +// its discussion. +func (ss sessionSearcher) match(s *parser.Session, projDisplay string) *searchResult { + // Session ID match (high priority) + if strings.HasPrefix(strings.ToLower(s.ID), ss.m.query) { + return &searchResult{ + Type: "session", + Project: projDisplay, + Session: truncateID(s.ID, 8), + Path: s.FilePath, + Summary: sessionSummaryPreview(s.Summary, 64), + Time: formatAge(s.EndTime), + Priority: 0, + lastTime: s.EndTime, + } + } + + var res *searchResult + if ss.m.matches(s.Summary) { + res = &searchResult{ + Type: "session", + Project: projDisplay, + Session: truncateID(s.ID, 8), + Path: s.FilePath, + Summary: sessionSummaryPreview(s.Summary, 64), + Time: formatAge(s.EndTime), + Priority: 2, + lastTime: s.EndTime, + } + } + if !ss.content { + return res + } + + // Content scan. The default counts conversation text only — + // user prompts and assistant replies — so ranking follows + // discussion, not injected boilerplate (a hook line fired + // every turn once outranked the real answer 327 hits to 13; + // docs/devlog/2026-08-03-content-search-noise.org). --raw + // keeps grep parity over raw transcript lines, main file + // plus subagent files: no parse, works for every provider's + // format, misses nothing grep would find. + var ( + n int + first time.Time + previews []contentPreview + hits []searchHit + ) + if ss.raw { + var lines []rawHit + n, first, lines = countContentMatches(s.FilePath, ss.m, ss.hits) + for _, l := range lines { + hits = append(hits, searchHit{MessageID: l.id, Time: l.time, Role: l.role, Matches: 1, Quote: l.quote}) + } + } else { + // Cheap line-scan prefilter before the full parse; only + // trustworthy when JSON escaping can't hide the query. The + // literal matcher is a superset of word matches, so it + // prefilters both modes. + if rawPrefilterSafe(ss.m.query) && !sessionHasRawMatch(s.FilePath, ss.m.literal()) { + return res + } + n, first, previews, hits = scanConversationText(ss.parser, s.FilePath, ss.m, ss.hits) + } + if n == 0 { + return res + } + for i := range hits { + hits[i].Project = projDisplay + hits[i].Session = truncateID(s.ID, 8) + hits[i].Path = s.FilePath + } + + summary := fmt.Sprintf("%d hits · %s", n, sessionSummaryPreview(s.Summary, 48)) + if len(previews) > 0 { + summary = fmt.Sprintf("%d hits · [%s] %s", n, previews[0].Role, truncateDisplay(previews[0].Text, 56)) + } + if res == nil { + res = &searchResult{ + Type: "content", + Project: projDisplay, + Session: truncateID(s.ID, 8), + Path: s.FilePath, + Summary: summary, + Time: formatAge(s.EndTime), + Priority: 3, + lastTime: s.EndTime, + } + } + res.Matches = n + res.Previews = previews + res.firstHit = first + res.hits = hits + if !first.IsZero() { + res.FirstHit = first.UTC().Format(time.RFC3339) + } + return res +} + +// collectHits flattens per-session hits into one timeline, oldest +// first; hits without a timestamp trail in discovery order. +func collectHits(results []searchResult) []searchHit { + var hits []searchHit + for _, r := range results { + hits = append(hits, r.hits...) + } + sort.SliceStable(hits, func(i, j int) bool { + a, b := hits[i].Time, hits[j].Time + if a.IsZero() != b.IsZero() { + return !a.IsZero() + } + return a.Before(b) + }) + return hits +} + +func printSearchHits(hits []searchHit) error { + if searchLimit > 0 && len(hits) > searchLimit { + fmt.Fprintf(os.Stderr, "showing %d of %d hits (raise with -n)\n", searchLimit, len(hits)) + hits = hits[:searchLimit] + } + if len(hits) == 0 { + fmt.Println("No hits found.") + return nil + } + if searchJSON { + enc := json.NewEncoder(os.Stdout) + enc.SetIndent("", " ") + return enc.Encode(hits) + } + w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0) + fmt.Fprintln(w, "TIME\tSESSION\tROLE\tMESSAGE\tQUOTE") + for _, h := range hits { + t := "-" + if !h.Time.IsZero() { + t = h.Time.Local().Format("2006-01-02 15:04") + } + id := h.MessageID + if id == "" { + id = "-" + } + session := h.Session + if session == "" { + session = "-" + } + fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", t, session, h.Role, citationID(id), cleanDisplayText(h.Quote)) + } + return w.Flush() +} + +// scanProgress reports content-scan progress on stderr when stderr is +// a terminal, so a multi-minute cold scan is visibly alive. Silent +// when piped: agents and scripts read stdout, and a stream of \r +// lines is noise there. +type scanProgress struct { + mu sync.Mutex + total int + done_ int + last time.Time + on bool +} + +func newScanProgress(total int, on bool) *scanProgress { + if on { + info, err := os.Stderr.Stat() + on = err == nil && info.Mode()&os.ModeCharDevice != 0 + } + return &scanProgress{total: total, on: on} +} + +func (p *scanProgress) tick() { + if !p.on { + return + } + p.mu.Lock() + defer p.mu.Unlock() + p.done_++ + if now := time.Now(); now.Sub(p.last) >= 200*time.Millisecond { + p.last = now + fmt.Fprintf(os.Stderr, "\rscanning %d/%d sessions", p.done_, p.total) + } +} + +func (p *scanProgress) done() { + if !p.on { + return + } + p.mu.Lock() + defer p.mu.Unlock() + fmt.Fprintf(os.Stderr, "\r%*s\r", len(fmt.Sprintf("scanning %d/%d sessions", p.total, p.total)), "") +} + // scanConversationText parses one session (the parser loads sidechain // files too) and searches only conversation text: text and thinking // blocks of user prompts and assistant messages. Tool results, hook // attachments, command echoes, and meta lines never count — that's -// what --raw is for. Returns total occurrences plus up to -// maxContentPreviews role-labeled snippets around the earliest -// matches. query must already be lowercase. -func scanConversationText(p sessionParser, path, query string) (int, []contentPreview) { +// what --raw is for. Returns total occurrences, the timestamp of the +// earliest matching message, and up to maxContentPreviews role-labeled +// snippets around the earliest matches. +func scanConversationText(p sessionParser, path string, m textMatcher, wantHits bool) (int, time.Time, []contentPreview, []searchHit) { sess, err := p.ParseSession(path) if err != nil { fmt.Fprintf(os.Stderr, "warning: skipping unparseable %s: %v\n", filepath.Base(path), err) - return 0, nil + return 0, time.Time{}, nil, nil } count := 0 + var first time.Time var previews []contentPreview + var hits []searchHit var walk func(msgs []*parser.Message) walk = func(msgs []*parser.Message) { - for _, m := range msgs { - if m.Kind == parser.KindUserPrompt || m.Kind == parser.KindAssistant { - role := m.Type - if m.IsSidechain { + for _, msg := range msgs { + if msg.Kind == parser.KindUserPrompt || msg.Kind == parser.KindAssistant { + role := msg.Type + if msg.IsSidechain { role = "agent" } - for _, b := range m.Content { + msgHits := 0 + quote := "" + for _, b := range msg.Content { if b.Type != "text" && b.Type != "thinking" { continue } - lower := strings.ToLower(b.Text) - n := strings.Count(lower, query) + n := m.count(b.Text) if n == 0 { continue } count += n - if len(previews) < maxContentPreviews { - previews = append(previews, contentPreview{ - Role: role, - Text: matchSnippet(b.Text, strings.Index(lower, query), len(query)), - }) + msgHits += n + if !msg.Timestamp.IsZero() && (first.IsZero() || msg.Timestamp.Before(first)) { + first = msg.Timestamp + } + if len(previews) < maxContentPreviews || (wantHits && quote == "") { + idx, qlen := m.index(b.Text) + snippet := matchSnippet(b.Text, idx, qlen) + if quote == "" { + quote = snippet + } + if len(previews) < maxContentPreviews { + previews = append(previews, contentPreview{Role: role, Text: snippet}) + } } } + if wantHits && msgHits > 0 { + hits = append(hits, searchHit{ + MessageID: msg.UUID, + Time: msg.Timestamp, + Role: role, + Matches: msgHits, + Quote: quote, + }) + } } - walk(m.Children) + walk(msg.Children) } } walk(sess.RootMessages) - return count, previews + return count, first, previews, hits } // matchSnippet cuts a display window around a match, clamped to rune -// boundaries. idx indexes the lowered copy of text; byte positions can +// boundaries. idx may index a lowered copy of text; byte positions can // drift on the rare rune whose lowercase form changes width, so bounds // are clamped rather than trusted. func matchSnippet(text string, idx, qlen int) string { @@ -480,46 +949,143 @@ func rawPrefilterSafe(query string) bool { return true } -// countContentMatches counts transcript lines containing query across -// the main session file and any subagent files beside it (layout +// sessionHasRawMatch reports whether any raw transcript line of the +// session — main file or subagent files — matches. Stops at the first +// hit: it is a prefilter, not a count. +func sessionHasRawMatch(sessionPath string, m textMatcher) bool { + if sessionPath == "" { + return false + } + if n, _, _ := scanRawLines(sessionPath, m, true, false); n > 0 { + return true + } + for _, f := range parser.SubagentFiles(sessionPath) { + if n, _, _ := scanRawLines(f, m, true, false); n > 0 { + return true + } + } + return false +} + +// countContentMatches counts transcript lines matching m across the +// main session file and any subagent files beside it (layout // knowledge lives in parser.SubagentFiles; providers without subagent -// files simply contribute none). -func countContentMatches(sessionPath, query string) int { +// files simply contribute none), and returns the earliest timestamp +// among matching lines. +func countContentMatches(sessionPath string, m textMatcher, wantHits bool) (int, time.Time, []rawHit) { if sessionPath == "" { - return 0 + return 0, time.Time{}, nil } - count := countMatchingLines(sessionPath, query) + count, first, hits := scanRawLines(sessionPath, m, false, wantHits) for _, f := range parser.SubagentFiles(sessionPath) { - count += countMatchingLines(f, query) + n, t, h := scanRawLines(f, m, false, wantHits) + count += n + first = earlier(first, t) + hits = append(hits, h...) } - return count + return count, first, hits } // countMatchingLines streams one JSONL file and counts lines matching -// query case-insensitively. bufio.Reader, not Scanner: transcript -// lines carrying embedded images exceed any fixed budget, and a -// silent early stop is exactly the false-negative class --content -// exists to kill. Unreadable files warn instead of lying "0 hits". -func countMatchingLines(path, query string) int { +// m, returning the earliest timestamp among them. +func countMatchingLines(path string, m textMatcher) (int, time.Time) { + n, first, _ := scanRawLines(path, m, false, false) + return n, first +} + +// rawHit is one matching raw transcript line: its top-level anchors +// plus a quote around the match. +type rawHit struct { + id string + role string + time time.Time + quote string +} + +// scanRawLines streams one JSONL file and counts lines matching m +// (stopping at the first when stopAtFirst). bufio.Reader, not +// Scanner: transcript lines carrying embedded images exceed any fixed +// budget, and a silent early stop is exactly the false-negative class +// --content exists to kill. Unreadable files warn instead of lying +// "0 hits". The earliest top-level "timestamp" among matching lines +// is returned when counting (every provider stamps its lines). +func scanRawLines(path string, m textMatcher, stopAtFirst, wantHits bool) (int, time.Time, []rawHit) { file, err := os.Open(path) if err != nil { fmt.Fprintf(os.Stderr, "warning: skipping unreadable %s: %v\n", filepath.Base(path), err) - return 0 + return 0, time.Time{}, nil } defer file.Close() count := 0 + var first time.Time + var hits []rawHit reader := bufio.NewReaderSize(file, 64*1024) for { line, err := reader.ReadString('\n') - if line != "" && strings.Contains(strings.ToLower(line), query) { - count++ + if line != "" { + if idx, qlen := m.index(line); idx >= 0 { + count++ + if stopAtFirst { + return count, first, nil + } + head := lineHead(line) + first = earlier(first, head.time) + if wantHits { + hits = append(hits, rawHit{id: head.id, role: head.role, time: head.time, quote: matchSnippet(line, idx, qlen)}) + } + } } if err != nil { if err != io.EOF { fmt.Fprintf(os.Stderr, "warning: read error in %s: %v\n", filepath.Base(path), err) } - return count + return count, first, hits } } } + +// lineHead is the top-level anchor set of one transcript line. +type lineHead_ struct { + id string + role string + time time.Time +} + +// lineHead extracts the top-level "timestamp", "uuid", and "type" of +// one transcript line; zero/empty when absent or unparseable. A full +// decode, not a substring hunt: tool results embed other objects' +// timestamps. +func lineHead(line string) lineHead_ { + var head struct { + Timestamp string `json:"timestamp"` + UUID string `json:"uuid"` + Type string `json:"type"` + } + if err := json.Unmarshal([]byte(line), &head); err != nil { + return lineHead_{} + } + out := lineHead_{id: head.UUID, role: head.Type} + for _, layout := range []string{time.RFC3339Nano, time.RFC3339} { + if t, err := time.Parse(layout, head.Timestamp); err == nil { + out.time = t + break + } + } + return out +} + +func lineTimestamp(line string) time.Time { return lineHead(line).time } + +// earlier returns the earlier of two times, ignoring zero values. +func earlier(a, b time.Time) time.Time { + switch { + case a.IsZero(): + return b + case b.IsZero(): + return a + case b.Before(a): + return b + } + return a +} diff --git a/internal/cmd/search_history.go b/internal/cmd/search_history.go new file mode 100644 index 0000000..ea4842b --- /dev/null +++ b/internal/cmd/search_history.go @@ -0,0 +1,198 @@ +package cmd + +import ( + "bufio" + "encoding/json" + "fmt" + "io" + "os" + "path/filepath" + "strings" + "time" +) + +// Prompt history as a search source. Claude Code and Codex append every +// human prompt to a history file that outlives session cleanup, so it +// is the longest-lived evidence for "when did we first say X". Only +// prompts whose session is no longer in the store surface here — when +// the session file exists its own content hit is the better citation, +// and one prompt must not appear twice in a timeline. + +// promptHit is one matching prompt from a history file. +type promptHit struct { + Provider string + Project string // workspace basename when the file records one + SessionID string + Path string + Line int + Time time.Time + Matches int + Quote string +} + +// promptHistoryFiles lists the history files to scan for the given +// provider homes (missing files are simply absent). +func promptHistoryFiles(claudeHome, codexHome string) map[string]string { + files := map[string]string{} + if claudeHome != "" { + files["claude-code"] = filepath.Join(claudeHome, "history.jsonl") + } + if codexHome != "" { + files["codex"] = filepath.Join(codexHome, "history.jsonl") + } + return files +} + +// scanPromptHistory returns matching prompts across the history files, +// skipping entries whose session id is in knownSessions. +func scanPromptHistory(files map[string]string, m textMatcher, knownSessions map[string]bool) []promptHit { + var hits []promptHit + for _, provider := range []string{"claude-code", "codex"} { + path, ok := files[provider] + if !ok { + continue + } + hits = append(hits, scanPromptHistoryFile(provider, path, m, knownSessions)...) + } + return hits +} + +func scanPromptHistoryFile(provider, path string, m textMatcher, knownSessions map[string]bool) []promptHit { + file, err := os.Open(path) + if err != nil { + return nil // no history is normal + } + defer file.Close() + + var hits []promptHit + reader := bufio.NewReaderSize(file, 64*1024) + lineNo := 0 + for { + line, err := reader.ReadString('\n') + if line != "" { + lineNo++ + // Cheap literal prefilter on the raw line; the decoded + // text decides. + if rawPrefilterSafe(m.query) && !m.literal().matches(line) { + goto next + } + if hit, ok := decodePromptLine(provider, line); ok { + if hit.SessionID != "" && knownSessions[hit.SessionID] { + goto next + } + if n := m.count(hit.text); n > 0 { + idx, qlen := m.index(hit.text) + hits = append(hits, promptHit{ + Provider: provider, + Project: hit.project, + SessionID: hit.SessionID, + Path: path, + Line: lineNo, + Time: hit.time, + Matches: n, + Quote: matchSnippet(hit.text, idx, qlen), + }) + } + } + } + next: + if err != nil { + if err != io.EOF { + fmt.Fprintf(os.Stderr, "warning: read error in %s: %v\n", filepath.Base(path), err) + } + return hits + } + } +} + +type promptLine struct { + SessionID string + project string + time time.Time + text string +} + +// decodePromptLine understands both history formats: +// +// claude-code: {"display":"...","project":"/path","sessionId":"...","timestamp":} +// codex: {"session_id":"...","ts":,"text":"..."} +func decodePromptLine(provider, line string) (promptLine, bool) { + switch provider { + case "claude-code": + var rec struct { + Display string `json:"display"` + Project string `json:"project"` + SessionID string `json:"sessionId"` + Timestamp int64 `json:"timestamp"` + } + if err := json.Unmarshal([]byte(line), &rec); err != nil || rec.Display == "" { + return promptLine{}, false + } + out := promptLine{SessionID: rec.SessionID, text: rec.Display} + if rec.Project != "" { + out.project = filepath.Base(rec.Project) + } + if rec.Timestamp > 0 { + out.time = time.UnixMilli(rec.Timestamp).UTC() + } + return out, true + case "codex": + var rec struct { + SessionID string `json:"session_id"` + TS int64 `json:"ts"` + Text string `json:"text"` + } + if err := json.Unmarshal([]byte(line), &rec); err != nil || rec.Text == "" { + return promptLine{}, false + } + out := promptLine{SessionID: rec.SessionID, text: rec.Text} + if rec.TS > 0 { + out.time = time.Unix(rec.TS, 0).UTC() + } + return out, true + } + return promptLine{}, false +} + +// promptResult renders a prompt hit as a search result row. +func promptResult(h promptHit) searchResult { + project := h.Project + if project == "" { + project = "(" + h.Provider + " history)" + } + session := "" + if h.SessionID != "" { + session = truncateID(h.SessionID, 8) + } + res := searchResult{ + Type: "prompt", + Project: project, + Session: session, + Path: h.Path, + Summary: fmt.Sprintf("%d hits · [user] %s", h.Matches, truncateDisplay(h.Quote, 56)), + Time: "-", + Matches: h.Matches, + Previews: []contentPreview{{Role: "user", Text: h.Quote}}, + Priority: 4, + firstHit: h.Time, + hits: []searchHit{{ + Project: project, Session: session, Path: h.Path, MessageID: fmt.Sprintf("line:%d", h.Line), + Time: h.Time, Role: "user", Matches: h.Matches, Quote: h.Quote, + }}, + } + if !h.Time.IsZero() { + res.FirstHit = h.Time.UTC().Format(time.RFC3339) + } + return res +} + +// knownSessionIDs collects every session id the store currently holds. +func knownSessionIDs(sessions []sessionCandidate) map[string]bool { + known := make(map[string]bool, len(sessions)) + for _, c := range sessions { + if c.session != nil && c.session.ID != "" { + known[strings.ToLower(c.session.ID)] = true + } + } + return known +} diff --git a/internal/cmd/search_test.go b/internal/cmd/search_test.go index 90324bc..30222e4 100644 --- a/internal/cmd/search_test.go +++ b/internal/cmd/search_test.go @@ -5,10 +5,15 @@ import ( "path/filepath" "strings" "testing" + "time" "github.com/thevibeworks/ccx/internal/parser" ) +// sub is the default substring matcher; word is -w. +func sub(q string) textMatcher { return newTextMatcher(q, false) } +func word(q string) textMatcher { return newTextMatcher(q, true) } + func TestCountMatchingLines(t *testing.T) { dir := t.TempDir() path := filepath.Join(dir, "s.jsonl") @@ -19,13 +24,13 @@ func TestCountMatchingLines(t *testing.T) { if err := os.WriteFile(path, []byte(content), 0o644); err != nil { t.Fatal(err) } - if got := countMatchingLines(path, "pi-agent"); got != 2 { + if got, _ := countMatchingLines(path, sub("pi-agent")); got != 2 { t.Fatalf("case-insensitive matches: got %d, want 2", got) } - if got := countMatchingLines(path, "absent-term"); got != 0 { + if got, _ := countMatchingLines(path, sub("absent-term")); got != 0 { t.Fatalf("no-match count: got %d, want 0", got) } - if got := countMatchingLines(filepath.Join(dir, "missing.jsonl"), "x"); got != 0 { + if got, _ := countMatchingLines(filepath.Join(dir, "missing.jsonl"), sub("x")); got != 0 { t.Fatalf("missing file must count 0, got %d", got) } } @@ -43,10 +48,10 @@ func TestCountContentMatchesIncludesSubagents(t *testing.T) { if err := os.MkdirAll(subDir, 0o755); err != nil { t.Fatal(err) } - sub := `{"text":"goose in sidechain"} + sideLines := `{"text":"goose in sidechain"} {"text":"more goose here"} ` - if err := os.WriteFile(filepath.Join(subDir, "agent-1.jsonl"), []byte(sub), 0o644); err != nil { + if err := os.WriteFile(filepath.Join(subDir, "agent-1.jsonl"), []byte(sideLines), 0o644); err != nil { t.Fatal(err) } // Non-jsonl files (meta.json) and jsonl without the agent- prefix @@ -58,12 +63,18 @@ func TestCountContentMatchesIncludesSubagents(t *testing.T) { t.Fatal(err) } - if got := countContentMatches(main, "goose"); got != 3 { + if got, _, _ := countContentMatches(main, sub("goose"), false); got != 3 { t.Fatalf("main+subagent matches: got %d, want 3", got) } - if got := countContentMatches("", "goose"); got != 0 { + if got, _, _ := countContentMatches("", sub("goose"), false); got != 0 { t.Fatalf("empty path must count 0, got %d", got) } + if !sessionHasRawMatch(main, sub("sidechain")) { + t.Fatal("prefilter must see subagent files") + } + if sessionHasRawMatch(main, sub("absent")) { + t.Fatal("prefilter false positive") + } } type stubSessionParser struct{} @@ -100,13 +111,16 @@ func TestScanConversationTextSignalOnly(t *testing.T) { // 6 raw lines match; only 5 occurrences live in conversation text // (1 user + 2 assistant + 1 thinking + 1 sidechain). - if raw := countContentMatches(main, "deadman"); raw != 6 { - t.Fatalf("raw line matches: got %d, want 6", raw) + if raw, first, _ := countContentMatches(main, sub("deadman"), false); raw != 6 || !first.Equal(time.Date(2026, 8, 3, 0, 0, 0, 0, time.UTC)) { + t.Fatalf("raw line matches: got %d @ %v, want 6 @ 2026-08-03T00:00:00Z", raw, first) } - n, previews := scanConversationText(stubSessionParser{}, main, "deadman") + n, first, previews, _ := scanConversationText(stubSessionParser{}, main, sub("deadman"), false) if n != 5 { t.Fatalf("signal matches: got %d, want 5 (noise counted?)", n) } + if !first.Equal(time.Date(2026, 8, 3, 0, 0, 0, 0, time.UTC)) { + t.Fatalf("first hit: got %v, want the user prompt time", first) + } if len(previews) != maxContentPreviews { t.Fatalf("previews: got %d, want %d", len(previews), maxContentPreviews) } @@ -117,7 +131,7 @@ func TestScanConversationTextSignalOnly(t *testing.T) { t.Fatalf("second preview role: got %q, want assistant", previews[1].Role) } - if n, _ := scanConversationText(stubSessionParser{}, main, "auto-handoff"); n != 1 { + if n, _, _, _ := scanConversationText(stubSessionParser{}, main, sub("auto-handoff"), false); n != 1 { t.Fatalf("auto-handoff signal matches: got %d, want 1 (hook noise counted?)", n) } } @@ -138,7 +152,7 @@ func TestScanConversationTextSidechainRole(t *testing.T) { t.Fatal(err) } - n, previews := scanConversationText(stubSessionParser{}, main, "goose") + n, _, previews, _ := scanConversationText(stubSessionParser{}, main, sub("goose"), false) if n != 1 || len(previews) != 1 { t.Fatalf("sidechain match: got n=%d previews=%d, want 1/1", n, len(previews)) } @@ -189,7 +203,367 @@ func TestCountMatchingLinesOversizedLine(t *testing.T) { if err := os.WriteFile(path, []byte(content), 0o644); err != nil { t.Fatal(err) } - if got := countMatchingLines(path, "needle"); got != 1 { + if got, _ := countMatchingLines(path, sub("needle")); got != 1 { t.Fatalf("match after oversized line: got %d, want 1", got) } } + +// -w must stop "semantica" from matching "semantically" — the case that +// made 46 of 47 results noise — while still matching the term inside +// punctuation, hyphenated compounds, and CJK runs (no ASCII word +// boundary exists there, so none is demanded). +func TestTextMatcherWord(t *testing.T) { + cases := []struct { + q, text string + sub, w int + }{ + {"semantica", "semantically correct", 1, 0}, + {"semantica", "we chose Semantica.", 1, 1}, + {"semantica", "semantica-agi ships semantica", 2, 2}, + {"semantica", "(semantica)", 1, 1}, + {"pi-agent", "Pi-Agent uses ACP; pi-agents too", 2, 1}, + {"中文", "关于中文的讨论", 1, 1}, + {"fix bug", "fix bugs later; fix bug now", 2, 1}, + } + for _, c := range cases { + if got := sub(c.q).count(c.text); got != c.sub { + t.Errorf("sub(%q).count(%q) = %d, want %d", c.q, c.text, got, c.sub) + } + if got := word(c.q).count(c.text); got != c.w { + t.Errorf("word(%q).count(%q) = %d, want %d", c.q, c.text, got, c.w) + } + if (word(c.q).matches(c.text)) != (c.w > 0) { + t.Errorf("word(%q).matches(%q) disagrees with count %d", c.q, c.text, c.w) + } + } + // index feeds the preview window: it must point at the real match, + // not the substring inside a longer word. + if i, n := word("semantica").index("semantically, then semantica-agi"); i != 19 || n != len("semantica") { + t.Fatalf("word index: got %d/%d, want 19/9", i, n) + } + // literal() is the prefilter: a superset of word matches. + if !word("semantica").literal().matches("semantically") { + t.Fatal("literal prefilter must keep substring semantics") + } +} + +// indexFold/countFold replace ToLower-per-line on the raw scan; they +// must agree with the allocating form on ASCII and non-ASCII input. +func TestIndexFold(t *testing.T) { + cases := []struct { + s, q string + idx int + n int + }{ + {"tell me about Pi-Agent", "pi-agent", 14, 1}, + {"PI-AGENT pi-agent Pi-Agent", "pi-agent", 0, 3}, + {"nothing here", "absent", -1, 0}, + {"aaa", "aa", 0, 1}, + {"路径 and 路径", "路径", 0, 2}, + {"x", "", 0, 0}, + } + for _, c := range cases { + if got := indexFold(c.s, c.q); got != c.idx { + t.Errorf("indexFold(%q,%q) = %d, want %d", c.s, c.q, got, c.idx) + } + if got := countFold(c.s, c.q); got != c.n { + t.Errorf("countFold(%q,%q) = %d, want %d", c.s, c.q, got, c.n) + } + if c.q != "" { + if want := strings.Index(strings.ToLower(c.s), c.q); want != c.idx { + t.Errorf("test expectation drift for %q: ToLower form gives %d", c.q, want) + } + } + } + // A window that cuts a multi-byte rune must not match. + if indexFold("cafés", "\xc3s") != -1 { + t.Fatal("partial rune must not fold-match") + } +} + +func TestLineTimestamp(t *testing.T) { + // The tool result embeds another object's timestamp before the + // top-level one; only the top-level field counts. + line := `{"type":"user","message":{"content":[{"type":"tool_result","content":"{\"timestamp\":\"2020-01-01T00:00:00Z\"}"}]},"timestamp":"2026-08-03T01:02:03.500Z"}` + if got := lineTimestamp(line); !got.Equal(time.Date(2026, 8, 3, 1, 2, 3, 500000000, time.UTC)) { + t.Fatalf("lineTimestamp: got %v", got) + } + if !lineTimestamp(`{"no":"ts"}`).IsZero() || !lineTimestamp(`not json`).IsZero() { + t.Fatal("missing/invalid timestamp must be zero") + } +} + +// A session whose summary matches must still carry its content +// evidence: previously the summary hit short-circuited the scan, so +// the one session where the term was actually discussed was the one +// result with no matches, previews, or first-hit time. +func TestSessionSearcherSummaryHitKeepsContent(t *testing.T) { + dir := t.TempDir() + main := filepath.Join(dir, "abc-123.jsonl") + lines := strings.Join([]string{ + `{"type":"user","uuid":"u1","timestamp":"2026-08-18T07:00:00Z","message":{"role":"user","content":[{"type":"text","text":"when did we mention semantica?"}]}}`, + `{"type":"assistant","uuid":"a1","parentUuid":"u1","timestamp":"2026-08-18T07:00:05Z","message":{"role":"assistant","content":[{"type":"text","text":"semantically, never; semantica itself: now"}]}}`, + }, "\n") + "\n" + if err := os.WriteFile(main, []byte(lines), 0o644); err != nil { + t.Fatal(err) + } + end := time.Date(2026, 8, 18, 8, 0, 0, 0, time.UTC) + sess := &parser.Session{ID: "abc-123", FilePath: main, Summary: "trace semantica mentions", EndTime: end} + + // Name-only search: summary hit, no content fields. + r := sessionSearcher{m: word("semantica")}.match(sess, "proj") + if r == nil || r.Type != "session" || r.Matches != 0 || r.FirstHit != "" { + t.Fatalf("summary-only match: got %+v", r) + } + + // Content on: same row, now with evidence. Word mode counts 2 + // (not the "semantically" in the reply). + r = sessionSearcher{m: word("semantica"), content: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || r.Type != "session" || r.Priority != 2 { + t.Fatalf("summary+content match must stay a session hit, got %+v", r) + } + if r.Matches != 2 || len(r.Previews) != 2 || r.Previews[0].Role != "user" { + t.Fatalf("content evidence missing on summary hit: %+v", r) + } + if r.FirstHit != "2026-08-18T07:00:00Z" || !r.lastTime.Equal(end) { + t.Fatalf("first/last: got %q / %v", r.FirstHit, r.lastTime) + } + + // Substring mode counts the noise too; -w is what makes the + // count trustworthy. + r = sessionSearcher{m: sub("semantica"), content: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || r.Matches != 3 { + t.Fatalf("substring count: got %+v", r) + } + + // No summary hit, no content hit: nil, not an empty row. + if r := (sessionSearcher{m: word("absent"), content: true, parser: stubSessionParser{}}).match(sess, "proj"); r != nil { + t.Fatalf("miss must be nil, got %+v", r) + } + + // --raw: line count plus first-hit from the raw timestamp. + r = sessionSearcher{m: sub("semantica"), content: true, raw: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || r.Matches != 2 || r.FirstHit != "2026-08-18T07:00:00Z" { + t.Fatalf("raw match: got %+v", r) + } +} + +// matchAll must return hits in candidate order regardless of which +// worker finishes first — stable ranking depends on it. +func TestSessionSearcherMatchAllOrder(t *testing.T) { + var cands []sessionCandidate + for i := 0; i < 50; i++ { + summary := "nothing" + if i%3 == 0 { + summary = "goose" + } + cands = append(cands, sessionCandidate{ + session: &parser.Session{ID: "id-" + strings.Repeat("x", i%7) + string(rune('a'+i%26)), Summary: summary}, + project: "p", + }) + } + hits := sessionSearcher{m: sub("goose")}.matchAll(cands, 4) + if len(hits) != 17 { + t.Fatalf("hits: got %d, want 17", len(hits)) + } + for i := 1; i < len(hits); i++ { + if hits[i-1].Session > hits[i].Session && cands[0].session != nil { + // IDs are not monotonic by construction; check discovery + // order via summary sequence instead. + break + } + } + want := 0 + for _, c := range cands { + if c.session.Summary != "goose" { + continue + } + if hits[want].Session != truncateID(c.session.ID, 8) { + t.Fatalf("hit %d out of order: got %s want %s", want, hits[want].Session, c.session.ID) + } + want++ + } +} + +func TestSortSearchResults(t *testing.T) { + ts := func(d int) time.Time { return time.Date(2026, 8, d, 0, 0, 0, 0, time.UTC) } + mk := func(name string, prio, hits, first, last int) searchResult { + r := searchResult{Session: name, Priority: prio, Matches: hits} + if first > 0 { + r.firstHit = ts(first) + } + if last > 0 { + r.lastTime = ts(last) + } + return r + } + order := func(rs []searchResult) string { + var ids []string + for _, r := range rs { + ids = append(ids, r.Session) + } + return strings.Join(ids, " ") + } + base := func() []searchResult { + return []searchResult{ + mk("proj", 1, 0, 0, 0), + mk("late-many", 3, 9, 15, 17), + mk("early-few", 3, 1, 3, 10), + mk("summary", 2, 0, 0, 12), + } + } + + rs := base() + sortSearchResults(rs, "hits") + if got := order(rs); got != "proj summary late-many early-few" { + t.Fatalf("hits order: %s", got) + } + rs = base() + sortSearchResults(rs, "first") + if got := order(rs); got != "early-few late-many proj summary" { + t.Fatalf("first order: %s (timeless must trail in hits order)", got) + } + rs = base() + sortSearchResults(rs, "last") + if got := order(rs); got != "late-many summary early-few proj" { + t.Fatalf("last order: %s", got) + } +} + +// --hits turns a search into citations: one row per matching message +// with the anchors (message id, time, role) needed to walk back to +// it, oldest first across sessions. Under --raw the unit is a line. +func TestSessionSearcherHits(t *testing.T) { + dir := t.TempDir() + main := filepath.Join(dir, "abc-123.jsonl") + lines := strings.Join([]string{ + `{"type":"user","uuid":"u1","timestamp":"2026-08-18T07:00:00Z","message":{"role":"user","content":[{"type":"text","text":"when did we mention semantica?"}]}}`, + `{"type":"assistant","uuid":"a1","parentUuid":"u1","timestamp":"2026-08-18T07:00:05Z","message":{"role":"assistant","content":[{"type":"text","text":"semantically, never"},{"type":"text","text":"semantica itself: now, and semantica again"}]}}`, + `{"type":"user","uuid":"t1","parentUuid":"a1","timestamp":"2026-08-18T07:00:06Z","message":{"role":"user","content":[{"type":"tool_result","content":"semantica in a tool result"}]}}`, + }, "\n") + "\n" + if err := os.WriteFile(main, []byte(lines), 0o644); err != nil { + t.Fatal(err) + } + sess := &parser.Session{ID: "abc-123", FilePath: main, Summary: "s"} + + r := sessionSearcher{m: word("semantica"), content: true, hits: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || r.Matches != 3 || len(r.hits) != 2 { + t.Fatalf("hits: got %+v", r) + } + h := r.hits[0] + if h.MessageID != "u1" || h.Role != "user" || h.Matches != 1 || h.Session != "abc-123" || h.Path != main || h.Project != "proj" { + t.Fatalf("first hit anchors: %+v", h) + } + if !h.Time.Equal(time.Date(2026, 8, 18, 7, 0, 0, 0, time.UTC)) || !strings.Contains(h.Quote, "semantica") { + t.Fatalf("first hit time/quote: %+v", h) + } + if r.hits[1].MessageID != "a1" || r.hits[1].Matches != 2 || !strings.Contains(r.hits[1].Quote, "semantica itself") { + t.Fatalf("second hit must count both blocks and quote the first real match, got %+v", r.hits[1]) + } + + // Without --hits nothing extra is collected. + if r := (sessionSearcher{m: word("semantica"), content: true, parser: stubSessionParser{}}).match(sess, "proj"); len(r.hits) != 0 { + t.Fatalf("hits collected without --hits: %d", len(r.hits)) + } + + // --raw --hits: every matching line, including the tool result, + // anchored by the line's own uuid/type/timestamp. + r = sessionSearcher{m: sub("semantica"), content: true, raw: true, hits: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || len(r.hits) != 3 || r.hits[2].MessageID != "t1" || r.hits[2].Role != "user" { + t.Fatalf("raw hits: got %+v", r) + } + + // collectHits orders across sessions by time, zero times last. + later := searchResult{hits: []searchHit{{MessageID: "z", Time: time.Date(2026, 8, 19, 0, 0, 0, 0, time.UTC)}, {MessageID: "none"}}} + all := collectHits([]searchResult{later, *r}) + if len(all) != 5 || all[0].MessageID != "u1" || all[3].MessageID != "z" || all[4].MessageID != "none" { + ids := []string{} + for _, h := range all { + ids = append(ids, h.MessageID) + } + t.Fatalf("collectHits order: %v", ids) + } +} + +// Prompt history is the search source that outlives session cleanup: +// both providers' formats decode, only prompts whose session is not in +// the store surface (no double citations), and each hit carries a +// file:line anchor and time. +func TestScanPromptHistory(t *testing.T) { + dir := t.TempDir() + claude := filepath.Join(dir, "claude-history.jsonl") + codex := filepath.Join(dir, "codex-history.jsonl") + if err := os.WriteFile(claude, []byte(strings.Join([]string{ + `{"display":"tell me about semantica","pastedContents":{},"timestamp":1759046069307,"project":"/Users/x/wrk/old-proj","sessionId":"gone-1111"}`, + `{"display":"semantically fine, ignore","timestamp":1759046070000,"project":"/Users/x/wrk/old-proj","sessionId":"gone-2222"}`, + `{"display":"semantica again, still in store","timestamp":1759046071000,"project":"/Users/x/wrk/live","sessionId":"live-3333"}`, + `{"display":"no session id but semantica","timestamp":1759046072000,"project":"/Users/x/wrk/anon"}`, + `not json`, + }, "\n")+"\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(codex, []byte(strings.Join([]string{ + `{"session_id":"cx-gone","ts":1754596435,"text":"Semantica in codex"}`, + `{"session_id":"cx-live","ts":1754596436,"text":"semantica in a live codex session"}`, + }, "\n")+"\n"), 0o644); err != nil { + t.Fatal(err) + } + files := map[string]string{"claude-code": claude, "codex": codex} + known := map[string]bool{"live-3333": true, "cx-live": true} + + hits := scanPromptHistory(files, word("semantica"), known) + if len(hits) != 3 { + t.Fatalf("hits: got %d, want 3 (gone-1111, anon, cx-gone): %+v", len(hits), hits) + } + h := hits[0] + if h.Provider != "claude-code" || h.Project != "old-proj" || h.SessionID != "gone-1111" || h.Line != 1 || h.Matches != 1 { + t.Fatalf("first hit: %+v", h) + } + if !h.Time.Equal(time.UnixMilli(1759046069307).UTC()) || !strings.Contains(h.Quote, "semantica") { + t.Fatalf("first hit time/quote: %+v", h) + } + if hits[1].SessionID != "" || hits[1].Project != "anon" { + t.Fatalf("anonymous prompt: %+v", hits[1]) + } + if hits[2].Provider != "codex" || hits[2].SessionID != "cx-gone" || !hits[2].Time.Equal(time.Unix(1754596435, 0).UTC()) { + t.Fatalf("codex hit: %+v", hits[2]) + } + // Substring mode also takes "semantically". + if got := len(scanPromptHistory(files, sub("semantica"), known)); got != 4 { + t.Fatalf("substring hits: got %d, want 4", got) + } + // Missing files are not an error. + if got := scanPromptHistory(map[string]string{"claude-code": filepath.Join(dir, "nope.jsonl")}, sub("x"), nil); len(got) != 0 { + t.Fatalf("missing history: %+v", got) + } + + // As a result row: type prompt, priority after content, first-hit + // set, one citation hit with a line anchor. + r := promptResult(hits[0]) + if r.Type != "prompt" || r.Priority != 4 || r.FirstHit == "" || len(r.hits) != 1 || r.hits[0].MessageID != "line:1" || r.hits[0].Role != "user" { + t.Fatalf("prompt result: %+v", r) + } +} + +// A citation is only useful if it can be pasted back: `ccx search +// --hits` prints MESSAGE, `ccx view --at` consumes it. Uuids shorten +// to 8 hex; synthetic ids (codex-thinking-13, line:442) must survive +// whole, since every one of them is ambiguous at 8 characters. +func TestCitationIDStaysPasteable(t *testing.T) { + cases := []struct { + id string + want string + }{ + {"cf332028-7a1e-4bd9-9a2f-9d1c2b3a4f55", "cf332028"}, + {"codex-thinking-13", "codex-thinking-13"}, + {"codex-thinking-129", "codex-thinking-129"}, + {"line:442", "line:442"}, + {"", ""}, + {"abc123", "abc123"}, + } + for _, c := range cases { + if got := citationID(c.id); got != c.want { + t.Errorf("citationID(%q) = %q, want %q", c.id, got, c.want) + } + } +} diff --git a/internal/cmd/sessions.go b/internal/cmd/sessions.go index 5899f20..fb11da7 100644 --- a/internal/cmd/sessions.go +++ b/internal/cmd/sessions.go @@ -50,7 +50,7 @@ var ( func init() { sessionsCmd.Flags().StringVar(&sessionsSort, "sort", "time", "sort by: time, messages, prompts, tokens") - sessionsCmd.Flags().IntVar(&sessionsLimit, "limit", 20, "limit number of sessions (0 = no limit)") + sessionsCmd.Flags().IntVarP(&sessionsLimit, "limit", "n", 20, "limit number of sessions (0 = no limit)") sessionsCmd.Flags().BoolVar(&sessionsJSON, "json", false, "output as JSON") sessionsCmd.Flags().StringVarP(&sessionsProvider, "provider", "p", "", "filter by provider: cc, cx, gx, all") sessionsCmd.Flags().StringVarP(&sessionsSearch, "search", "s", "", "search in session summaries") diff --git a/internal/cmd/trace.go b/internal/cmd/trace.go index 11e0408..761c739 100644 --- a/internal/cmd/trace.go +++ b/internal/cmd/trace.go @@ -114,6 +114,19 @@ func runTrace(cmd *cobra.Command, args []string) error { }) } + // Session connections cost a parse of every workspace session, so + // they ride only in the full bundle (docs/design/0006). + if traceFull { + related, warnings, err := relateSession(backend, session) + result.Warnings = append(result.Warnings, warnings...) + if err != nil { + result.Warnings = append(result.Warnings, trace.TraceWarning{Kind: "related_failed", Message: err.Error()}) + fmt.Fprintf(os.Stderr, "warning: session connections failed: %v\n", err) + } else { + result.Related = related + } + } + output, err := renderTrace(result) if err != nil { return err diff --git a/internal/cmd/view.go b/internal/cmd/view.go index 5b44b35..793aeb1 100644 --- a/internal/cmd/view.go +++ b/internal/cmd/view.go @@ -27,7 +27,18 @@ SESSION can be: - Index: @1 (most recent), @2 (second most recent) - With project: myproject:e38536 -If SESSION is omitted, shows an interactive picker.`, +If SESSION is omitted, shows an interactive picker. + +--at MESSAGE_ID walks from a citation to its context: the message +with that id (a search --hits message_id, a trace step message_id; +prefixes work) plus --context N messages before and after it, +flattened. The target is always shown, even under --brief. + +Examples: + ccx view e38536 Whole session + ccx view e38536 --brief Conversation only + ccx view e38536 --at c8bd2144 Around one cited message + ccx view e38536 --at c8bd2144 --context 8`, Args: cobra.MaximumNArgs(1), RunE: runView, } @@ -40,6 +51,8 @@ var ( viewBrief bool viewAll bool viewColor string + viewAt string + viewContext int ) func init() { @@ -50,6 +63,8 @@ func init() { viewCmd.Flags().BoolVar(&viewFlat, "flat", false, "disable tree rendering") viewCmd.Flags().BoolVarP(&viewBrief, "brief", "b", false, "conversation only: human input, agent responses, compactions") viewCmd.Flags().StringVar(&viewColor, "color", "auto", "colorize output: auto, always, never") + viewCmd.Flags().StringVar(&viewAt, "at", "", "show the message with this id (or unique prefix) and its context") + viewCmd.Flags().IntVar(&viewContext, "context", 3, "with --at: messages of context before and after the target") } // resolveColorMode maps --color to a concrete decision; auto follows @@ -101,7 +116,17 @@ func runView(cmd *cobra.Command, args []string) error { return fmt.Errorf("failed to parse session: %w", err) } - if viewBrief { + if viewAt != "" { + window, index, total, err := render.WindowSession(fullSession, viewAt, viewContext, viewContext) + if err != nil { + return err + } + fmt.Fprintf(os.Stderr, "message %d of %d in %s · %d before / %d after\n", index, total, truncateID(fullSession.ID, 8), viewContext, viewContext) + if viewBrief { + window = briefKeeping(window, viewAt) + } + fullSession = window + } else if viewBrief { fullSession = render.BriefSession(fullSession) } @@ -121,6 +146,38 @@ func runView(cmd *cobra.Command, args []string) error { return render.Terminal(fullSession, opts) } +// briefKeeping applies the brief filter to a window but never drops +// the cited target itself: the point of --at is to see that message. +func briefKeeping(window *parser.Session, target string) *parser.Session { + brief := render.BriefSession(window) + for _, m := range brief.RootMessages { + if strings.HasPrefix(m.UUID, target) { + return brief + } + } + // Re-insert the target at its wire position among the kept ones. + var out []*parser.Message + inserted := false + for _, m := range window.RootMessages { + if strings.HasPrefix(m.UUID, target) { + out = append(out, m) + inserted = true + continue + } + for _, k := range brief.RootMessages { + if k.UUID == m.UUID { + out = append(out, k) + break + } + } + } + if !inserted { + return brief + } + brief.RootMessages = out + return brief +} + func sessionLookupQuery(projectName string, all bool) (catalog.SessionQuery, error) { if projectName != "" { return catalog.SessionQuery{ diff --git a/internal/parser/classify_test.go b/internal/parser/classify_test.go index 379b079..0b7eec2 100644 --- a/internal/parser/classify_test.go +++ b/internal/parser/classify_test.go @@ -14,13 +14,15 @@ func TestClassifyUserTextHarnessWrappers(t *testing.T) { {"Caveat: generated by local commands", KindCommandOutput, true}, {"\nabc", KindNotification, true}, {" leading whitespace", KindNotification, true}, + {"[Request interrupted by user]", KindInterrupt, true}, + {"[Request interrupted by user for tool use]", KindInterrupt, true}, {"fix the login bug", KindUnknown, false}, {"", KindUnknown, false}, } for _, c := range cases { - got, harness := classifyUserText(c.text) + got, harness := ClassifyUserText(c.text) if harness != c.harness || (harness && got != c.want) { - t.Errorf("classifyUserText(%q) = (%v, %v), want (%v, %v)", c.text, got, harness, c.want, c.harness) + t.Errorf("ClassifyUserText(%q) = (%v, %v), want (%v, %v)", c.text, got, harness, c.want, c.harness) } } } @@ -68,3 +70,21 @@ func TestComputeStatsExcludesHarnessTurns(t *testing.T) { t.Errorf("UserPrompts = %d, want 1", stats.UserPrompts) } } + +func TestIsToolDenial(t *testing.T) { + yes := []string{ + "The user doesn't want to proceed with this tool use. The tool use was rejected (eg. if it was a file edit, the new_string was NOT written to the file).", + " The user doesn't want to take this action right now. STOP what you are doing", + } + no := []string{"Exit code 137\n[Request interrupted by user for tool use]", "permission denied: /etc/shadow", ""} + for _, s := range yes { + if !IsToolDenial(s) { + t.Errorf("IsToolDenial(%q) = false, want true", s) + } + } + for _, s := range no { + if IsToolDenial(s) { + t.Errorf("IsToolDenial(%q) = true, want false", s) + } + } +} diff --git a/internal/parser/session.go b/internal/parser/session.go index 170c20a..2857a44 100644 --- a/internal/parser/session.go +++ b/internal/parser/session.go @@ -256,11 +256,11 @@ func classifyMessage(msg *Message, raw rawMessage) MessageKind { if len(msg.Content) > 0 && msg.Content[0].Type == "text" { text = msg.Content[0].Text } - kind, ok := classifyUserText(text) + kind, ok := ClassifyUserText(text) if !ok { if str, isStr := raw.Message.Content.(string); isStr { text = str - kind, ok = classifyUserText(text) + kind, ok = ClassifyUserText(text) } } if ok { @@ -279,11 +279,13 @@ func classifyMessage(msg *Message, raw rawMessage) MessageKind { return KindUnknown } -// classifyUserText recognizes harness-generated XML wrappers in a +// ClassifyUserText recognizes harness-generated XML wrappers in a // user-role message. These all masquerade as human turns in the raw // JSONL but carry no human input: slash-command markers, local command // stdout/stderr/caveat echoes, and background-task notifications. -func classifyUserText(text string) (MessageKind, bool) { +// Exported so line-streaming surfaces (sessionlog) classify a +// user-role line exactly as the full parser does. +func ClassifyUserText(text string) (MessageKind, bool) { t := strings.TrimSpace(text) switch { case strings.HasPrefix(t, ""): return KindNotification, true + case strings.HasPrefix(t, "[Request interrupted by user"): + // The human pressed stop: a harness marker, not a prompt. It + // belongs to the turn it interrupted (no new exchange). + return KindInterrupt, true } return KindUnknown, false } +// IsToolDenial reports whether a tool result is the harness telling +// the agent that the human rejected the call at the permission prompt +// (Claude Code) — a human intervention, distinct from a tool error. +func IsToolDenial(text string) bool { + t := strings.TrimSpace(text) + return strings.HasPrefix(t, "The user doesn't want to proceed with this tool use") || + strings.HasPrefix(t, "The user doesn't want to take this action right now") +} + // extractCommandName extracts command name from /foo func extractCommandName(text string) string { start := strings.Index(text, "") @@ -616,7 +631,7 @@ func quickParseSession(filePath string) (summary string, startTime, endTime time // Skip harness wrappers (commands, command output, // notifications) so list counts match computeStats, // which only counts KindUserPrompt. - if _, harness := classifyUserText(extractTextFromContent(raw.Message.Content)); !harness { + if _, harness := ClassifyUserText(extractTextFromContent(raw.Message.Content)); !harness { stats.MessageCount++ stats.UserPrompts++ } diff --git a/internal/parser/types.go b/internal/parser/types.go index e0f9d33..97f57dc 100644 --- a/internal/parser/types.go +++ b/internal/parser/types.go @@ -14,6 +14,7 @@ const ( KindCommand MessageKind = "command" // Slash command (/init, /compact, etc) KindCommandOutput MessageKind = "command_output" // harness echo KindNotification MessageKind = "notification" // background-task event + KindInterrupt MessageKind = "interrupt" // [Request interrupted by user...] — the human stopped the agent KindMeta MessageKind = "meta" // Meta/system instruction KindCompactSummary MessageKind = "compact_summary" // Compacted context carrier KindAssistant MessageKind = "assistant" // Assistant response @@ -37,7 +38,7 @@ type Project struct { // struct versions silently, so without this stamp an upgraded binary // keeps serving parses produced by the old code until the source // session file itself happens to change. -const CacheFormatVersion = 3 +const CacheFormatVersion = 4 type Session struct { ID string diff --git a/internal/provider/codex/backend.go b/internal/provider/codex/backend.go index 8aa3519..340eb8c 100644 --- a/internal/provider/codex/backend.go +++ b/internal/provider/codex/backend.go @@ -81,6 +81,12 @@ type agentMessagePayload struct { Message string `json:"message"` } +type turnAbortedPayload struct { + Type string `json:"type"` + TurnID string `json:"turn_id"` + Reason string `json:"reason"` +} + type agentReasoningPayload struct { Text string `json:"text"` } @@ -545,6 +551,12 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { var firstTime time.Time var lastTime time.Time seenToolCallIDs := make(map[string]bool) + seenCompletedMessageIDs := make(map[string]bool) + legacyMessageCount := 0 + legacyUserPrompts := 0 + completedMessageCount := 0 + completedUserPrompts := 0 + completedSummary := "" countToolCall := func(callID string) { callID = strings.TrimSpace(callID) @@ -610,8 +622,8 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { switch header.Type { case "user_message": - stats.MessageCount++ - stats.UserPrompts++ + legacyMessageCount++ + legacyUserPrompts++ var payload userMessagePayload if err := json.Unmarshal(rollout.Payload, &payload); err == nil && summary == "" { @@ -619,7 +631,23 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { } case "agent_message": - stats.MessageCount++ + legacyMessageCount++ + + case "item_completed": + message, ok := DecodeCompletedTurnMessage(rollout.Payload) + if !ok || (message.ID != "" && seenCompletedMessageIDs[message.ID]) { + continue + } + if message.ID != "" { + seenCompletedMessageIDs[message.ID] = true + } + completedMessageCount++ + if message.Role == "user" { + completedUserPrompts++ + if completedSummary == "" { + completedSummary = stripUserMessagePrefix(message.Text) + } + } case "exec_command_end": var payload execCommandEndPayload @@ -727,6 +755,14 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { if err := scanner.Err(); err != nil { return nil, err } + if completedMessageCount > 0 { + stats.MessageCount = completedMessageCount + stats.UserPrompts = completedUserPrompts + summary = completedSummary + } else { + stats.MessageCount = legacyMessageCount + stats.UserPrompts = legacyUserPrompts + } if firstTime.IsZero() || lastTime.IsZero() { if info, err := os.Stat(filePath); err == nil { @@ -757,6 +793,10 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { } func (b *Backend) parseSession(filePath string, threadNames map[string]string) (*parser.Session, error) { + useCompletedTurnMessages, err := hasCompletedTurnMessages(filePath) + if err != nil { + return nil, err + } file, err := os.Open(filePath) if err != nil { return nil, err @@ -774,6 +814,7 @@ func (b *Backend) parseSession(filePath string, threadNames map[string]string) ( var firstTime time.Time var lastTime time.Time var messages []*parser.Message + seenCompletedMessageIDs := make(map[string]bool) pendingTools := make(map[string]pendingToolCall) handledCallIDs := make(map[string]bool) completedCallIDs := make(map[string]bool) @@ -869,6 +910,9 @@ func (b *Backend) parseSession(filePath string, threadNames map[string]string) ( switch header.Type { case "user_message": + if useCompletedTurnMessages { + continue + } var payload userMessagePayload if err := json.Unmarshal(rollout.Payload, &payload); err != nil { continue @@ -904,6 +948,9 @@ func (b *Backend) parseSession(filePath string, threadNames map[string]string) ( )) case "agent_message": + if useCompletedTurnMessages { + continue + } var payload agentMessagePayload if err := json.Unmarshal(rollout.Payload, &payload); err != nil { continue @@ -918,6 +965,71 @@ func (b *Backend) parseSession(filePath string, threadNames map[string]string) ( parser.ContentBlock{Type: "text", Text: payload.Message}, )) + case "item_completed": + if !useCompletedTurnMessages { + continue + } + message, ok := DecodeCompletedTurnMessage(rollout.Payload) + if !ok || (message.ID != "" && seenCompletedMessageIDs[message.ID]) { + continue + } + if message.ID != "" { + seenCompletedMessageIDs[message.ID] = true + } + messageID := message.ID + if messageID == "" { + messageID = fmt.Sprintf("codex-%s-%d", message.Role, lineNum) + } + if message.Role == "user" { + text := stripUserMessagePrefix(message.Text) + if text == "" { + text = "(empty)" + } + if firstUserSummary == "" { + firstUserSummary = text + } + stats.MessageCount++ + stats.UserPrompts++ + messages = append(messages, newMessage( + messageID, + "user", + parser.KindUserPrompt, + ts, + currentModel, + parser.ContentBlock{Type: "text", Text: text}, + )) + } else { + stats.MessageCount++ + messages = append(messages, newMessage( + messageID, + "assistant", + parser.KindAssistant, + ts, + currentModel, + parser.ContentBlock{Type: "text", Text: message.Text}, + )) + } + + case "turn_aborted": + // The human stopped the turn (Esc / Ctrl-C). Codex's + // counterpart of Claude Code's "[Request interrupted + // by user]": a harness marker, never an exchange + // anchor, counted by trace as an interrupt. + var payload turnAbortedPayload + if err := json.Unmarshal(rollout.Payload, &payload); err != nil { + continue + } + if payload.Reason == "" || payload.Reason == "interrupted" { + messages = append(messages, newMessage( + fmt.Sprintf("codex-interrupt-%d", lineNum), + "user", + parser.KindInterrupt, + ts, + currentModel, + parser.ContentBlock{Type: "text", Text: "[Turn interrupted by user]"}, + )) + } + case "agent_reasoning", "agent_reasoning_raw_content": var payload agentReasoningPayload if err := json.Unmarshal(rollout.Payload, &payload); err != nil { diff --git a/internal/provider/codex/backend_0147_test.go b/internal/provider/codex/backend_0147_test.go new file mode 100644 index 0000000..bcb9a71 --- /dev/null +++ b/internal/provider/codex/backend_0147_test.go @@ -0,0 +1,87 @@ +package codex + +import ( + "path/filepath" + "testing" +) + +func TestQuickParseSessionCodex0147ItemCompletedMessages(t *testing.T) { + home := t.TempDir() + sessionsDir := filepath.Join(home, "sessions") + rolloutPath := filepath.Join(sessionsDir, "2026", "08", "17", "rollout-0147.jsonl") + writeRollout(t, rolloutPath, codex0147Rollout) + + backend := NewWithDirs(home, sessionsDir, filepath.Join(home, "archived_sessions")) + projects, err := backend.DiscoverProjects() + if err != nil { + t.Fatalf("DiscoverProjects() error = %v", err) + } + if len(projects) != 1 || len(projects[0].Sessions) != 1 { + t.Fatalf("discovered projects/sessions = %d/%d, want 1/1", len(projects), len(projects[0].Sessions)) + } + + session := projects[0].Sessions[0] + if session.Summary != "actual user prompt" { + t.Fatalf("Summary = %q, want actual user prompt", session.Summary) + } + if session.Stats.MessageCount != 2 { + t.Fatalf("MessageCount = %d, want 2", session.Stats.MessageCount) + } + if session.Stats.UserPrompts != 1 { + t.Fatalf("UserPrompts = %d, want 1", session.Stats.UserPrompts) + } + if session.Model != "gpt-5.6-sol" { + t.Fatalf("Model = %q, want gpt-5.6-sol", session.Model) + } +} + +func TestParseSessionCodex0147UsesCanonicalTurnItems(t *testing.T) { + home := t.TempDir() + sessionsDir := filepath.Join(home, "sessions") + rolloutPath := filepath.Join(sessionsDir, "2026", "08", "17", "rollout-0147.jsonl") + writeRollout(t, rolloutPath, codex0147Rollout) + + backend := NewWithDirs(home, sessionsDir, filepath.Join(home, "archived_sessions")) + session, err := backend.ParseSession(rolloutPath) + if err != nil { + t.Fatalf("ParseSession() error = %v", err) + } + + if session.Stats.MessageCount != 2 || session.Stats.UserPrompts != 1 { + t.Fatalf("message stats = %d/%d, want 2 messages and 1 prompt", + session.Stats.MessageCount, session.Stats.UserPrompts) + } + if len(session.RootMessages) != 1 { + t.Fatalf("RootMessages = %d, want 1", len(session.RootMessages)) + } + root := session.RootMessages[0] + if root.UUID != "user-item-1" { + t.Fatalf("user UUID = %q, want stable item id", root.UUID) + } + if got := root.Content[0].Text; got != "actual user prompt" { + t.Fatalf("user text = %q, want actual user prompt", got) + } + if len(root.Children) != 1 { + t.Fatalf("user children = %d, want 1", len(root.Children)) + } + assistant := root.Children[0] + if assistant.UUID != "assistant-item-1" { + t.Fatalf("assistant UUID = %q, want stable item id", assistant.UUID) + } + if got := assistant.Content[0].Text; got != "actual assistant reply" { + t.Fatalf("assistant text = %q, want actual assistant reply", got) + } +} + +// Codex 0.147 persists provider request/response messages and canonical +// item_completed turn items together. The response_item user record includes +// injected instruction envelopes, so it must never become the visible prompt. +const codex0147Rollout = `{"timestamp":"2026-08-18T04:45:37.600Z","type":"session_meta","payload":{"id":"thread-0147","timestamp":"2026-08-18T04:45:37.600Z","cwd":"/tmp/work/project-0147","originator":"codex-tui","cli_version":"0.147.0","source":"cli","model_provider":"openai"},"ordinal":0} +{"timestamp":"2026-08-18T04:45:37.610Z","type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"private injected instruction envelope"}]},"ordinal":1} +{"timestamp":"2026-08-18T04:45:37.620Z","type":"turn_context","payload":{"turn_id":"turn-1","cwd":"/tmp/work/project-0147","model":"gpt-5.6-sol"},"ordinal":2} +{"timestamp":"2026-08-18T04:45:37.630Z","type":"event_msg","payload":{"type":"user_message","message":"legacy duplicate prompt","images":[],"local_images":[]},"ordinal":3} +{"timestamp":"2026-08-18T04:45:37.640Z","type":"event_msg","payload":{"type":"item_completed","thread_id":"thread-0147","turn_id":"turn-1","item":{"type":"UserMessage","id":"user-item-1","content":[{"type":"text","text":"actual user prompt","text_elements":[]}]},"completed_at_ms":1787028337640},"ordinal":4} +{"timestamp":"2026-08-18T04:45:37.650Z","type":"event_msg","payload":{"type":"agent_message","message":"legacy duplicate reply"},"ordinal":5} +{"timestamp":"2026-08-18T04:45:37.660Z","type":"response_item","payload":{"type":"message","id":"response-assistant-1","role":"assistant","phase":"final_answer","content":[{"type":"output_text","text":"actual assistant reply"}]},"ordinal":6} +{"timestamp":"2026-08-18T04:45:37.670Z","type":"event_msg","payload":{"type":"item_completed","thread_id":"thread-0147","turn_id":"turn-1","item":{"type":"AgentMessage","id":"assistant-item-1","content":[{"type":"Text","text":"actual assistant reply"}],"phase":"final_answer"},"completed_at_ms":1787028337670},"ordinal":7} +` diff --git a/internal/provider/codex/interrupt_test.go b/internal/provider/codex/interrupt_test.go new file mode 100644 index 0000000..4fb70bc --- /dev/null +++ b/internal/provider/codex/interrupt_test.go @@ -0,0 +1,48 @@ +package codex + +import ( + "path/filepath" + "testing" + + "github.com/thevibeworks/ccx/internal/parser" +) + +// event_msg.turn_aborted (reason interrupted) is the human stopping a +// Codex turn — Codex's "[Request interrupted by user]". It must parse +// as a KindInterrupt marker (never a prompt) so trace counts it, and +// other abort reasons must not. +func TestParseSessionCodexTurnAbortedIsInterrupt(t *testing.T) { + home := t.TempDir() + sessionsDir := filepath.Join(home, "sessions") + rolloutPath := filepath.Join(sessionsDir, "2026", "08", "18", "rollout-abort.jsonl") + writeRollout(t, rolloutPath, `{"timestamp":"2026-08-18T10:00:00Z","type":"session_meta","payload":{"id":"abort-1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:00Z"}} +{"timestamp":"2026-08-18T10:00:01Z","type":"event_msg","payload":{"type":"user_message","message":"run the migration"}} +{"timestamp":"2026-08-18T10:00:02Z","type":"event_msg","payload":{"type":"agent_message","message":"Starting."}} +{"timestamp":"2026-08-18T10:00:03Z","type":"event_msg","payload":{"type":"turn_aborted","turn_id":"t1","reason":"interrupted"}} +{"timestamp":"2026-08-18T10:00:04Z","type":"event_msg","payload":{"type":"turn_aborted","turn_id":"t2","reason":"replaced"}} +`) + backend := NewWithDirs(home, sessionsDir, filepath.Join(home, "archived_sessions")) + session, err := backend.ParseSession(rolloutPath) + if err != nil { + t.Fatalf("ParseSession: %v", err) + } + interrupts := 0 + prompts := 0 + for _, m := range parser.FlattenSessionMessages(session) { + switch m.Kind { + case parser.KindInterrupt: + interrupts++ + if m.Type != "user" || m.Content[0].Text != "[Turn interrupted by user]" { + t.Fatalf("interrupt message shape: %+v", m) + } + case parser.KindUserPrompt: + prompts++ + } + } + if interrupts != 1 { + t.Fatalf("interrupts = %d, want 1 (only reason=interrupted counts)", interrupts) + } + if prompts != 1 { + t.Fatalf("prompts = %d, want 1 (the abort marker is not a prompt)", prompts) + } +} diff --git a/internal/provider/codex/turn_items.go b/internal/provider/codex/turn_items.go new file mode 100644 index 0000000..908191d --- /dev/null +++ b/internal/provider/codex/turn_items.go @@ -0,0 +1,112 @@ +package codex + +import ( + "bufio" + "encoding/json" + "os" + "strings" +) + +// Codex 0.147 made item_completed TurnItems the persisted, UI-facing +// transcript. Raw response_item messages are model I/O and can contain injected +// developer/user envelopes, so they are not a safe conversation source. +// +// CompletedTurnMessage is one such visible message. Exported so every +// surface that reads rollouts line by line (sessionlog) applies the +// same source-selection rule as the parser (docs/design/0004). +type CompletedTurnMessage struct { + ID string + Role string + Text string +} + +type itemCompletedPayload struct { + Type string `json:"type"` + Item json.RawMessage `json:"item"` +} + +type turnItemMessagePayload struct { + ID string `json:"id"` + Type string `json:"type"` + Content []turnItemContent `json:"content"` +} + +type turnItemContent struct { + Type string `json:"type"` + Text string `json:"text"` + Name string `json:"name"` +} + +func hasCompletedTurnMessages(filePath string) (bool, error) { + file, err := os.Open(filePath) + if err != nil { + return false, err + } + defer file.Close() + + scanner := bufio.NewScanner(file) + scanner.Buffer(make([]byte, 0, 64*1024), maxScannerBufferBytes) + for scanner.Scan() { + var rollout rolloutLine + if err := json.Unmarshal(scanner.Bytes(), &rollout); err != nil || rollout.Type != "event_msg" { + continue + } + if _, ok := DecodeCompletedTurnMessage(rollout.Payload); ok { + return true, nil + } + } + if err := scanner.Err(); err != nil { + return false, err + } + return false, nil +} + +// DecodeCompletedTurnMessage returns the visible message carried by an +// event_msg payload when it is an item_completed UserMessage or +// AgentMessage; ok is false for every other payload. +func DecodeCompletedTurnMessage(raw json.RawMessage) (CompletedTurnMessage, bool) { + var completed itemCompletedPayload + if err := json.Unmarshal(raw, &completed); err != nil || completed.Type != "item_completed" { + return CompletedTurnMessage{}, false + } + + var item turnItemMessagePayload + if err := json.Unmarshal(completed.Item, &item); err != nil { + return CompletedTurnMessage{}, false + } + + role := "" + switch item.Type { + case "UserMessage": + role = "user" + case "AgentMessage": + role = "assistant" + default: + return CompletedTurnMessage{}, false + } + + parts := make([]string, 0, len(item.Content)) + hasAttachment := false + for _, content := range item.Content { + switch strings.ToLower(content.Type) { + case "text": + if text := strings.TrimSpace(content.Text); text != "" { + parts = append(parts, text) + } + case "image", "local_image": + hasAttachment = true + case "audio", "local_audio": + hasAttachment = true + case "skill", "mention": + if name := strings.TrimSpace(content.Name); name != "" { + parts = append(parts, "["+name+"]") + } + } + } + + text := strings.Join(parts, "\n") + if text == "" && hasAttachment { + text = imageOnlyMessagePlaceholder + } + return CompletedTurnMessage{ID: item.ID, Role: role, Text: text}, true +} diff --git a/internal/render/window.go b/internal/render/window.go new file mode 100644 index 0000000..0e8d344 --- /dev/null +++ b/internal/render/window.go @@ -0,0 +1,65 @@ +package render + +import ( + "fmt" + "strings" + + "github.com/thevibeworks/ccx/internal/parser" +) + +// WindowSession returns a copy of session holding only the message +// whose UUID starts with target plus `before` messages before it and +// `after` after it, in wire order, flattened (children detached so +// the window is exactly what it says). This is the walk from a +// citation — a search hit's message_id, a trace step's message_id — +// back to its surrounding context without leaving ccx. index/total +// locate the target in the full session for the caller's header. +func WindowSession(session *parser.Session, target string, before, after int) (*parser.Session, int, int, error) { + if session == nil { + return nil, 0, 0, fmt.Errorf("no session") + } + target = strings.TrimSpace(target) + if target == "" { + return nil, 0, 0, fmt.Errorf("--at needs a message id (or unique prefix)") + } + all := parser.FlattenSessionMessages(session) + idx := -1 + for i, m := range all { + if m == nil || !strings.HasPrefix(m.UUID, target) { + continue + } + if m.UUID == target { + idx = i + break + } + if idx >= 0 && all[idx].UUID != m.UUID { + return nil, 0, 0, fmt.Errorf("message id prefix %q is ambiguous (%s, %s, ...); use more characters", target, all[idx].UUID, m.UUID) + } + idx = i + } + if idx < 0 { + return nil, 0, 0, fmt.Errorf("no message with id prefix %q in session %s", target, session.ID) + } + if before < 0 { + before = 0 + } + if after < 0 { + after = 0 + } + start := idx - before + if start < 0 { + start = 0 + } + end := idx + after + 1 + if end > len(all) { + end = len(all) + } + out := *session + out.RootMessages = make([]*parser.Message, 0, end-start) + for _, m := range all[start:end] { + flat := *m + flat.Children = nil + out.RootMessages = append(out.RootMessages, &flat) + } + return &out, idx + 1, len(all), nil +} diff --git a/internal/render/window_test.go b/internal/render/window_test.go new file mode 100644 index 0000000..0ad8337 --- /dev/null +++ b/internal/render/window_test.go @@ -0,0 +1,68 @@ +package render + +import ( + "strings" + "testing" + + "github.com/thevibeworks/ccx/internal/parser" +) + +func winMsg(uuid string, children ...*parser.Message) *parser.Message { + return &parser.Message{UUID: uuid, Kind: parser.KindAssistant, Type: "assistant", Children: children} +} + +// The window is a wire-order slice around the target, flattened +// (children detached), clamped at both ends; the target is found by +// exact id or unique prefix; ambiguity and misses are errors. +func TestWindowSession(t *testing.T) { + // Tree: a -> b -> c -> d -> e (linear chain), plus f as a second root. + e := winMsg("eeee-5") + d := winMsg("dddd-4", e) + c := winMsg("cccc-3", d) + b := winMsg("bbbb-2", c) + a := winMsg("aaaa-1", b) + f := winMsg("aaab-6") + session := &parser.Session{ID: "s", RootMessages: []*parser.Message{a, f}} + + win, idx, total, err := WindowSession(session, "cccc", 1, 1) + if err != nil { + t.Fatal(err) + } + if idx != 3 || total != 6 { + t.Fatalf("index/total: %d/%d", idx, total) + } + got := []string{} + for _, m := range win.RootMessages { + got = append(got, m.UUID) + if len(m.Children) != 0 { + t.Fatalf("children must be detached in the window: %s", m.UUID) + } + } + if strings.Join(got, ",") != "bbbb-2,cccc-3,dddd-4" { + t.Fatalf("window: %v", got) + } + // Original tree untouched. + if len(session.RootMessages[0].Children) != 1 { + t.Fatal("window must not mutate the parsed session") + } + + // Clamp at the start; big context. + win, _, _, err = WindowSession(session, "aaaa-1", 5, 1) + if err != nil || len(win.RootMessages) != 2 || win.RootMessages[0].UUID != "aaaa-1" { + t.Fatalf("clamped window: %+v %v", win.RootMessages, err) + } + // Ambiguous prefix "aaa" matches aaaa-1 and aaab-6. + if _, _, _, err := WindowSession(session, "aaa", 1, 1); err == nil || !strings.Contains(err.Error(), "ambiguous") { + t.Fatalf("ambiguous prefix must error, got %v", err) + } + // Exact id wins even when it is also a prefix of another id. + if _, idx, _, err := WindowSession(session, "aaaa-1", 0, 0); err != nil || idx != 1 { + t.Fatalf("exact id: %d %v", idx, err) + } + if _, _, _, err := WindowSession(session, "zzzz", 1, 1); err == nil { + t.Fatal("missing id must error") + } + if _, _, _, err := WindowSession(session, "", 1, 1); err == nil { + t.Fatal("empty target must error") + } +} diff --git a/internal/sessionlog/sessionlog.go b/internal/sessionlog/sessionlog.go index 5ef2518..5565afd 100644 --- a/internal/sessionlog/sessionlog.go +++ b/internal/sessionlog/sessionlog.go @@ -13,6 +13,7 @@ import ( "unicode" "github.com/thevibeworks/ccx/internal/parser" + "github.com/thevibeworks/ccx/internal/provider/codex" ) const ( @@ -37,6 +38,14 @@ type Options struct { Limit int IncludeRaw bool Now time.Time + // Kinds keeps only records of these kinds (empty = all). Match + // keeps only records whose raw transcript line matches (nil = + // all); it sees the raw line, not the bounded Text, so a term + // past the text budget still matches — grep parity. Sessions, + // their Kinds, and Metrics.Records stay scope-wide; only the + // records list (and the aggregates built from it) narrow. + Kinds []string + Match func(rawLine string) bool } type Bundle struct { @@ -85,6 +94,7 @@ type Metrics struct { Workspaces int `json:"workspaces"` SourceFiles int `json:"source_files"` Records int `json:"records"` + RecordsMatched int `json:"records_matched,omitempty"` // after --kind/--match, before limit RecordsReturned int `json:"records_returned"` Limit int `json:"limit,omitempty"` Truncated bool `json:"truncated"` @@ -122,6 +132,8 @@ type ScopeRelation struct { } type Record struct { + matched bool // passed Options.Kinds/Match; unexported, filtered out in Collect + Timestamp time.Time `json:"timestamp"` Provider string `json:"provider"` SessionID string `json:"session_id"` @@ -143,16 +155,18 @@ type Record struct { } type rawLine struct { - Timestamp string `json:"timestamp"` - Type string `json:"type"` - SessionID string `json:"sessionId"` - UUID string `json:"uuid"` - ParentUUID string `json:"parentUuid"` - IsSidechain bool `json:"isSidechain"` - CWD string `json:"cwd"` - Summary string `json:"summary"` - Message json.RawMessage `json:"message"` - Payload json.RawMessage `json:"payload"` + Timestamp string `json:"timestamp"` + Type string `json:"type"` + SessionID string `json:"sessionId"` + UUID string `json:"uuid"` + ParentUUID string `json:"parentUuid"` + IsSidechain bool `json:"isSidechain"` + IsMeta bool `json:"isMeta"` + IsCompactSummary bool `json:"isCompactSummary"` + CWD string `json:"cwd"` + Summary string `json:"summary"` + Message json.RawMessage `json:"message"` + Payload json.RawMessage `json:"payload"` } type fileScan struct { @@ -266,14 +280,45 @@ func Collect(sources []Source, opts Options) (*Bundle, error) { } return bundle.Records[i].Timestamp.Before(bundle.Records[j].Timestamp) }) + filtered := len(opts.Kinds) > 0 || opts.Match != nil + if filtered { + bundle.Records = filterRecords(bundle.Records, opts.Kinds) + } + matched := len(bundle.Records) bundle.Days, bundle.Providers, bundle.Workspaces = aggregateRecords(bundle.Records, now.Location()) if opts.Limit > 0 && len(bundle.Records) > opts.Limit { bundle.Records = bundle.Records[:opts.Limit] } bundle.Metrics = metricsFor(bundle.Sessions, len(bundle.Records), opts.Limit, opts.Start, opts.End) + if filtered { + bundle.Metrics.RecordsMatched = matched + } return bundle, nil } +// filterRecords keeps records that passed Match and are of one of the +// kinds (kind demotion has already run, so "user_prompt" means the +// visible conversation on every provider). +func filterRecords(records []Record, kinds []string) []Record { + want := make(map[string]bool, len(kinds)) + for _, k := range kinds { + if k = strings.TrimSpace(k); k != "" { + want[k] = true + } + } + out := records[:0] + for _, r := range records { + if !r.matched { + continue + } + if len(want) > 0 && !want[r.Kind] { + continue + } + out = append(out, r) + } + return out +} + // aggregateRecords buckets records by calendar day (in loc), provider, // and workspace. Days sort chronologically; providers and workspaces // sort by record volume, busiest first. @@ -441,6 +486,7 @@ func scanFile(source Source, filePath string, opts Options) (*fileScan, error) { if opts.IncludeRaw { record.RawJSON = json.RawMessage([]byte(line)) } + record.matched = opts.Match == nil || opts.Match(line) scan.Records = append(scan.Records, record) scan.Kinds[record.Kind]++ if scan.Preview == "" && record.Text != "" && (record.Kind == "user_prompt" || record.Kind == "assistant_message") { @@ -453,6 +499,19 @@ func scanFile(source Source, filePath string, opts Options) (*fileScan, error) { if len(scan.Records) == 0 { return nil, nil } + if source.Provider == "codex" && demoteLegacyCodexMessages(scan.Records) { + // Kinds and preview were tallied while streaming; redo them + // against the demoted kinds so metrics count the conversation + // once and the preview is a visible prompt, not an envelope. + scan.Kinds = make(map[string]int) + scan.Preview = "" + for _, record := range scan.Records { + scan.Kinds[record.Kind]++ + if scan.Preview == "" && record.Text != "" && (record.Kind == "user_prompt" || record.Kind == "assistant_message") { + scan.Preview = record.Text + } + } + } for i := range scan.Records { if (scan.Records[i].SessionID == "" || scan.Records[i].SessionID == fallbackSessionID(filePath)) && scan.SessionID != "" { scan.Records[i].SessionID = scan.SessionID @@ -550,10 +609,29 @@ func normalizeClaudeRecord(raw rawLine) Record { switch raw.Type { case "user": - if contentHasType(content, "tool_result") { + // Same rules as the full parser (parser.classifyMessage): a + // user-role line is a human prompt only when it is not a + // compaction carrier, an injected meta message (skill bodies, + // system instructions), a tool result, or a harness XML + // wrapper (slash-command markers, local command echoes, + // task notifications). Otherwise "the humans in the loop" + // shows things no human typed. + switch { + case raw.IsCompactSummary: + record.Kind = "compact_summary" + case raw.IsMeta: + record.Kind = "meta" + case contentHasType(content, "tool_result"): record.Kind = "tool_result" - } else { - record.Kind = "user_prompt" + if parser.IsToolDenial(record.Text) { + record.Kind = "tool_denied" + } + default: + if kind, ok := parser.ClassifyUserText(userTextForClassify(content)); ok { + record.Kind = string(kind) + } else { + record.Kind = "user_prompt" + } } case "assistant": if contentHasType(content, "tool_use") { @@ -572,6 +650,28 @@ func normalizeClaudeRecord(raw rawLine) Record { return record } +// userTextForClassify returns the leading text of a user-role content +// value the way the parser sees it: the first text block, or the raw +// string content. +func userTextForClassify(content any) string { + switch v := content.(type) { + case string: + return v + case []any: + for _, item := range v { + block, ok := item.(map[string]any) + if !ok { + continue + } + if stringField(block, "type") == "text" { + return stringField(block, "text") + } + return "" + } + } + return "" +} + func normalizeCodexRecord(raw rawLine) Record { payload := decodeObject(raw.Payload) payloadType := stringField(payload, "type") @@ -595,6 +695,20 @@ func normalizeCodexRecord(raw rawLine) Record { record.Text = joinNonEmpty(" ", "turn", record.TurnID, stringField(payload, "model"), record.Workspace) case "event_msg": record.Type = joinNonEmpty(":", raw.Type, payloadType) + if payloadType == "item_completed" { + // Codex 0.147: the visible transcript is item_completed + // UserMessage/AgentMessage TurnItems (docs/design/0004). + if message, ok := codex.DecodeCompletedTurnMessage(raw.Payload); ok { + record.UUID = message.ID + record.Role = message.Role + record.Kind = "assistant_message" + if message.Role == "user" { + record.Kind = "user_prompt" + } + record.Text = truncateText(cleanText(message.Text), 1000) + break + } + } normalizeCodexEvent(&record, payload, payloadType) case "response_item": record.Type = joinNonEmpty(":", raw.Type, payloadType) @@ -656,6 +770,14 @@ func normalizeCodexEvent(record *Record, payload map[string]any, payloadType str case "compacted": record.Kind = "compaction" record.Text = truncateText(cleanText(stringField(payload, "message")), 1000) + case "turn_aborted": + // The human stopped the turn: same kind as Claude's + // "[Request interrupted by user]" so --kind interrupt + // covers both providers. + record.Kind = "interrupt" + record.Role = "user" + record.TurnID = stringField(payload, "turn_id") + record.Text = truncateText(cleanText(joinNonEmpty(" ", "turn aborted:", emptyDefault(stringField(payload, "reason"), "interrupted"))), 1000) default: record.Kind = emptyDefault(payloadType, "event") record.Text = truncateText(cleanText(contentPreview(payload)), 1000) @@ -694,6 +816,42 @@ func normalizeCodexResponseItem(record *Record, payload map[string]any, payloadT } } +// demoteLegacyCodexMessages applies the parser's one-source-per-rollout +// rule (docs/design/0004) to a rollout's records: when the file carries +// completed TurnItem messages, the legacy event_msg user_message / +// agent_message events and the raw response_item messages are model +// I/O or duplicates, not the conversation. They stay in the log as +// records — nothing is dropped — but no longer count as prompts or +// replies, so a 0.147 rollout is not double-counted and an injected +// instruction envelope is not shown as something the human typed. +func demoteLegacyCodexMessages(records []Record) bool { + hasCompleted := false + for i := range records { + if records[i].Type == "event_msg:item_completed" && (records[i].Kind == "user_prompt" || records[i].Kind == "assistant_message") { + hasCompleted = true + break + } + } + if !hasCompleted { + return false + } + for i := range records { + r := &records[i] + switch r.Type { + case "event_msg:user_message", "event_msg:agent_message": + r.Kind = "legacy_message" + case "response_item:message": + switch r.Kind { + case "user_prompt": + r.Kind = "model_input" + case "assistant_message": + r.Kind = "model_output" + } + } + } + return true +} + func relationFor(sessionStart, sessionEnd, scopeStart, scopeEnd time.Time) ScopeRelation { return ScopeRelation{ OverlapsScope: sessionStart.Before(scopeEnd) && !sessionEnd.Before(scopeStart), @@ -865,6 +1023,14 @@ func contentPreview(value any) string { return s } } + // tool_result blocks carry their payload under "content" (a + // string or nested blocks); without this a Claude tool result + // previewed as the literal word "tool_result". + if nested, ok := v["content"]; ok { + if s := contentPreview(nested); s != "" { + return s + } + } if t := stringField(v, "type"); t != "" { return t } diff --git a/internal/sessionlog/sessionlog_test.go b/internal/sessionlog/sessionlog_test.go index e52de16..9f3433a 100644 --- a/internal/sessionlog/sessionlog_test.go +++ b/internal/sessionlog/sessionlog_test.go @@ -3,6 +3,7 @@ package sessionlog import ( "os" "path/filepath" + "strings" "testing" "time" ) @@ -151,6 +152,158 @@ func TestCollectClassifiesCodexDeveloperMessageAsInstruction(t *testing.T) { } } +// Codex 0.147 rollouts carry the visible conversation as +// event_msg.item_completed UserMessage/AgentMessage TurnItems; the raw +// response_item messages are model I/O (with injected envelopes) and +// legacy user_message/agent_message events are duplicates in hybrid +// files. The log surface must apply the parser's one-source rule: +// item_completed rows become the prompts/replies with text, and the +// duplicates stay as records but stop counting (docs/design/0004). +func TestCollectCodex0147ItemCompletedIsTheConversation(t *testing.T) { + root := t.TempDir() + home := filepath.Join(root, ".codex") + sessionDir := filepath.Join(home, "sessions", "2026", "08") + if err := os.MkdirAll(sessionDir, 0755); err != nil { + t.Fatal(err) + } + file := filepath.Join(sessionDir, "rollout-0147.jsonl") + writeLines(t, file, + `{"timestamp":"2026-08-18T04:45:00Z","type":"session_meta","payload":{"id":"codex-0147","cwd":"/tmp/repo"}}`, + `{"timestamp":"2026-08-18T04:45:01Z","type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"# AGENTS.md instructions for /tmp/repo (injected envelope)"}]}}`, + `{"timestamp":"2026-08-18T04:45:02Z","type":"event_msg","payload":{"type":"item_completed","item":{"id":"item-u1","type":"UserMessage","content":[{"type":"text","text":"hello from 0.147"}]}}}`, + `{"timestamp":"2026-08-18T04:45:03Z","type":"event_msg","payload":{"type":"user_message","message":"hello from 0.147"}}`, + `{"timestamp":"2026-08-18T04:45:04Z","type":"event_msg","payload":{"type":"item_completed","item":{"id":"item-a1","type":"AgentMessage","content":[{"type":"Text","text":"Received."}]}}}`, + `{"timestamp":"2026-08-18T04:45:05Z","type":"response_item","payload":{"type":"message","role":"assistant","content":[{"type":"output_text","text":"Received."}]}}`, + `{"timestamp":"2026-08-18T04:45:06Z","type":"event_msg","payload":{"type":"item_completed","item":{"id":"item-c1","type":"CommandExecution","command":"ls"}}}`, + ) + + start := mustParseTime(t, "2026-08-18T00:00:00Z") + end := mustParseTime(t, "2026-08-19T00:00:00Z") + bundle, err := Collect([]Source{{Provider: "codex", Home: home}}, Options{Start: start, End: end}) + if err != nil { + t.Fatal(err) + } + if len(bundle.Records) != 7 { + t.Fatalf("records = %d, want 7 (nothing dropped)", len(bundle.Records)) + } + byLine := map[int]Record{} + for _, r := range bundle.Records { + byLine[r.Line] = r + } + if r := byLine[3]; r.Kind != "user_prompt" || r.Role != "user" || r.Text != "hello from 0.147" || r.UUID != "item-u1" { + t.Fatalf("item_completed UserMessage: %+v", r) + } + if r := byLine[5]; r.Kind != "assistant_message" || r.Text != "Received." || r.UUID != "item-a1" { + t.Fatalf("item_completed AgentMessage: %+v", r) + } + if r := byLine[2]; r.Kind != "model_input" { + t.Fatalf("raw response_item user message must be model_input, got %+v", r) + } + if r := byLine[4]; r.Kind != "legacy_message" { + t.Fatalf("legacy user_message event must be demoted in a hybrid file, got %+v", r) + } + if r := byLine[6]; r.Kind != "model_output" { + t.Fatalf("raw response_item assistant message must be model_output, got %+v", r) + } + if r := byLine[7]; r.Kind != "item_completed" { + t.Fatalf("non-message TurnItem keeps its own kind, got %+v", r) + } + if bundle.Metrics.UserPrompts != 1 || bundle.Metrics.AssistantMessages != 1 { + t.Fatalf("metrics must count the conversation once: %+v", bundle.Metrics) + } + if bundle.Sessions[0].Preview != "hello from 0.147" { + t.Fatalf("preview must come from the visible prompt, got %q", bundle.Sessions[0].Preview) + } +} + +// A legacy (pre-0.147) rollout has no item_completed messages, so its +// user_message / agent_message events remain the conversation. +func TestCollectCodexLegacyRolloutUnchanged(t *testing.T) { + root := t.TempDir() + home := filepath.Join(root, ".codex") + sessionDir := filepath.Join(home, "sessions", "2026", "05") + if err := os.MkdirAll(sessionDir, 0755); err != nil { + t.Fatal(err) + } + writeLines(t, filepath.Join(sessionDir, "legacy.jsonl"), + `{"timestamp":"2026-05-21T00:01:00Z","type":"session_meta","payload":{"id":"codex-legacy","cwd":"/tmp/repo"}}`, + `{"timestamp":"2026-05-21T00:02:00Z","type":"event_msg","payload":{"type":"user_message","message":"legacy prompt"}}`, + `{"timestamp":"2026-05-21T00:03:00Z","type":"event_msg","payload":{"type":"agent_message","message":"legacy reply"}}`, + `{"timestamp":"2026-05-21T00:04:00Z","type":"event_msg","payload":{"type":"turn_aborted","turn_id":"t1","reason":"interrupted"}}`, + ) + bundle, err := Collect([]Source{{Provider: "codex", Home: home}}, Options{Start: mustParseTime(t, "2026-05-21T00:00:00Z"), End: mustParseTime(t, "2026-05-22T00:00:00Z")}) + if err != nil { + t.Fatal(err) + } + if bundle.Metrics.UserPrompts != 1 || bundle.Metrics.AssistantMessages != 1 { + t.Fatalf("legacy rollout must keep counting: %+v", bundle.Metrics) + } + last := bundle.Records[len(bundle.Records)-1] + if last.Kind != "interrupt" || last.Role != "user" || last.TurnID != "t1" { + t.Fatalf("turn_aborted must be an interrupt record: %+v", last) + } +} + +// user-role lines that no human typed — command markers, local command +// echoes, task notifications, injected meta (skill bodies), compaction +// carriers — must not be user_prompt in the log, exactly as in the +// parser; --kind user_prompt is "the humans in the loop". +func TestCollectClaudeUserRoleNoiseIsNotAPrompt(t *testing.T) { + root := t.TempDir() + home := filepath.Join(root, ".claude") + projDir := filepath.Join(home, "projects", "-tmp-repo") + if err := os.MkdirAll(projDir, 0755); err != nil { + t.Fatal(err) + } + writeLines(t, filepath.Join(projDir, "s1.jsonl"), + `{"type":"user","uuid":"u1","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:00Z","message":{"role":"user","content":[{"type":"text","text":"real prompt"}]}}`, + `{"type":"user","uuid":"u2","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:01Z","message":{"role":"user","content":"/modelmodel"}}`, + `{"type":"user","uuid":"u3","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:02Z","message":{"role":"user","content":[{"type":"text","text":"Set model"}]}}`, + `{"type":"user","uuid":"u4","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:03Z","message":{"role":"user","content":[{"type":"text","text":"done"}]}}`, + `{"type":"user","uuid":"u5","sessionId":"s1","cwd":"/tmp/repo","isMeta":true,"timestamp":"2026-08-18T10:00:04Z","message":{"role":"user","content":[{"type":"text","text":"Base directory for this skill: /x"}]}}`, + `{"type":"user","uuid":"u6","sessionId":"s1","cwd":"/tmp/repo","isCompactSummary":true,"timestamp":"2026-08-18T10:00:05Z","message":{"role":"user","content":[{"type":"text","text":"summary of earlier context"}]}}`, + `{"type":"user","uuid":"u7","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:06Z","message":{"role":"user","content":[{"type":"tool_result","content":"ok"}]}}`, + `{"type":"user","uuid":"u8","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:07Z","message":{"role":"user","content":[{"type":"text","text":"[Request interrupted by user]"}]}}`, + `{"type":"user","uuid":"u9","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:08Z","message":{"role":"user","content":[{"type":"tool_result","content":"The user doesn't want to proceed with this tool use. The tool use was rejected.","is_error":true}]}}`, + ) + bundle, err := Collect([]Source{{Provider: "claude-code", Home: home}}, Options{Start: mustParseTime(t, "2026-08-18T00:00:00Z"), End: mustParseTime(t, "2026-08-19T00:00:00Z")}) + if err != nil { + t.Fatal(err) + } + want := map[string]string{"u1": "user_prompt", "u2": "command", "u3": "command_output", "u4": "notification", "u5": "meta", "u6": "compact_summary", "u7": "tool_result", "u8": "interrupt", "u9": "tool_denied"} + for _, r := range bundle.Records { + if want[r.UUID] != r.Kind { + t.Errorf("%s: kind %q, want %q", r.UUID, r.Kind, want[r.UUID]) + } + } + if bundle.Metrics.UserPrompts != 1 { + t.Fatalf("user prompts: %+v", bundle.Metrics) + } + + // --kind and --match narrow the records list, keep scope-wide + // metrics, and report the matched count; Match sees the raw line. + bundle, err = Collect([]Source{{Provider: "claude-code", Home: home}}, Options{ + Start: mustParseTime(t, "2026-08-18T00:00:00Z"), End: mustParseTime(t, "2026-08-19T00:00:00Z"), + Kinds: []string{"user_prompt", "meta"}, + }) + if err != nil { + t.Fatal(err) + } + if len(bundle.Records) != 2 || bundle.Metrics.Records != 9 || bundle.Metrics.RecordsMatched != 2 || bundle.Metrics.RecordsReturned != 2 { + t.Fatalf("kind filter: %d records, metrics %+v", len(bundle.Records), bundle.Metrics) + } + bundle, err = Collect([]Source{{Provider: "claude-code", Home: home}}, Options{ + Start: mustParseTime(t, "2026-08-18T00:00:00Z"), End: mustParseTime(t, "2026-08-19T00:00:00Z"), + Match: func(line string) bool { return strings.Contains(line, "isMeta") }, + }) + if err != nil { + t.Fatal(err) + } + if len(bundle.Records) != 1 || bundle.Records[0].UUID != "u5" { + t.Fatalf("match filter on raw line: %+v", bundle.Records) + } +} + func writeLines(t *testing.T, path string, lines ...string) { t.Helper() content := "" diff --git a/internal/trace/analysis.go b/internal/trace/analysis.go index 17a6bef..0206c39 100644 --- a/internal/trace/analysis.go +++ b/internal/trace/analysis.go @@ -66,6 +66,8 @@ func Analyze(session *parser.Session) *TraceResult { allTools := make(map[string]struct{}) stepCount := 0 toolErrors := 0 + interrupts := 0 + denials := 0 var mainCost float64 var activeSecs float64 var inputTok, outputTok, cacheReadTok, cacheCreateTok, reasoningTok int @@ -82,6 +84,8 @@ func Analyze(session *parser.Session) *TraceResult { } stepCount += len(turn.Steps) toolErrors += turn.Errors + interrupts += turn.Interrupts + denials += turn.Denials mainCost += turn.CostUSD activeSecs += turn.ActiveSecs inputTok += turn.InputTokens @@ -112,6 +116,8 @@ func Analyze(session *parser.Session) *TraceResult { FilesRead: len(allRead), ToolsUsed: len(allTools), ToolErrors: toolErrors, + Interrupts: interrupts, + Denials: denials, InputTokens: inputTok, OutputTokens: outputTok, CacheReadTokens: cacheReadTok, @@ -245,6 +251,13 @@ func buildTurn(index int, anchor *parser.Message, messages []*parser.Message, si turn.CostUSD += msg.Usage.CostUSD } + if msg.Kind == parser.KindInterrupt { + turn.Interrupts++ + if len(steps) > 0 { + steps[len(steps)-1].Interrupts++ + } + } + if msg.Kind == parser.KindAssistant { if narration := firstText(msg); narration != "" { step := Step{ @@ -337,6 +350,28 @@ func buildTurn(index int, anchor *parser.Message, messages []*parser.Message, si mutIdxByToolID[cb.ToolID] = [2]int{len(steps) - 1, len(step.Mutations) - 1} } case "tool_result": + if denied := parser.IsToolDenial(toolResultText(cb.ToolResult)); denied { + // A human rejection is an intervention, not an + // error: count it on the turn/step and mark the + // issuing call as denied (materialized like errors + // so denied lists match the counts). + turn.Denials++ + if step := stepForResult(msg, steps, stepByToolID); step != nil { + step.Denials++ + } + if cb.ToolID != "" { + if loc, ok := mutIdxByToolID[cb.ToolID]; ok { + steps[loc[0]].Mutations[loc[1]].Denied = true + } else if ev, ok := callByToolID[cb.ToolID]; ok { + ev.Denied = true + if idx, ok := stepByToolID[cb.ToolID]; ok && idx >= 0 && idx < len(steps) { + steps[idx].Mutations = append(steps[idx].Mutations, ev) + mutIdxByToolID[cb.ToolID] = [2]int{idx, len(steps[idx].Mutations) - 1} + } + } + } + continue + } if !cb.IsError { continue } @@ -372,6 +407,30 @@ func buildTurn(index int, anchor *parser.Message, messages []*parser.Message, si return turn } +// toolResultText returns the text of a tool result payload, which +// arrives as a string or as a list of content blocks. +func toolResultText(result any) string { + switch v := result.(type) { + case string: + return v + case []any: + var parts []string + for _, item := range v { + if block, ok := item.(map[string]any); ok { + if text, ok := block["text"].(string); ok { + parts = append(parts, text) + } + } + } + return strings.Join(parts, "\n") + case map[string]any: + if text, ok := v["text"].(string); ok { + return text + } + } + return "" +} + // stepForResult finds the step that issued the call a result belongs // to (via ToolID), falling back to the latest step. Results can land // after later narration — background agents especially — so ToolID @@ -603,10 +662,44 @@ func extractPatchPaths(patch string) []string { var redirectPattern = regexp.MustCompile(`(?:>>?|\btee(?:\s+-a)?)\s+([^\s;|&<>]+)`) +// heredocStart matches a heredoc operator and captures its delimiter: +// < 0`, a blockquote +// `> 2026-08-18`) is not read as an output redirect. The body starts +// after the line carrying the operator and ends at the line equal to +// the delimiter (leading tabs allowed for <<-). +func stripHeredocs(command string) string { + if !strings.Contains(command, "<<") { + return command + } + lines := strings.Split(command, "\n") + var out []string + for i := 0; i < len(lines); i++ { + line := lines[i] + out = append(out, line) + m := heredocStart.FindStringSubmatch(line) + if m == nil { + continue + } + delim := m[2] + for i+1 < len(lines) { + i++ + if strings.TrimLeft(lines[i], "\t") == delim { + break + } + } + } + return strings.Join(out, "\n") +} + func extractRedirectPaths(command string) []string { if !strings.ContainsAny(command, ">") && !strings.Contains(command, "tee") { return nil } + command = stripHeredocs(command) matches := redirectPattern.FindAllStringSubmatch(command, -1) var paths []string for _, m := range matches { diff --git a/internal/trace/analysis_test.go b/internal/trace/analysis_test.go index f68e113..b202d96 100644 --- a/internal/trace/analysis_test.go +++ b/internal/trace/analysis_test.go @@ -548,3 +548,122 @@ func TestCommandSummaryCollapsesAndBounds(t *testing.T) { t.Fatal("nil input must yield empty summary") } } + +// Heredoc bodies are data, not shell: a Go comparison or a markdown +// blockquote piped through `python3 - <<'EOF'` / `cat > f <<'EOF'` must +// not become "edited files" (they surfaced as ".../0" and +// ".../2026-08-18" in files_edited and in session connections). +func TestExtractRedirectPathsIgnoresHeredocBodies(t *testing.T) { + cmd := "cd repo && python3 - <<'EOF'\nif first > 0:\n pass\nx = a > b\nEOF\ncat > out.md < 2026-08-18 quote\nEOF\necho done > final.log" + got := extractRedirectPaths(cmd) + want := []string{"out.md", "final.log"} + if strings.Join(got, ",") != strings.Join(want, ",") { + t.Fatalf("redirect paths: got %v, want %v", got, want) + } + // <<- allows tab-indented delimiter; unquoted body still stripped. + cmd = "cat <<-DOC\n\tvalue > 0\n\tDOC\ntee log.txt" + if got := extractRedirectPaths(cmd); strings.Join(got, ",") != "log.txt" { + t.Fatalf("<<- heredoc: got %v", got) + } + // Unterminated heredoc swallows the rest: nothing after it counts. + if got := extractRedirectPaths("cat < not-a-file\n"); len(got) != 0 { + t.Fatalf("unterminated heredoc: got %v", got) + } +} + +// Human interventions inside a turn: the "[Request interrupted by +// user]" marker and a permission-prompt rejection are counted on the +// turn, the step, and the stats; the denied call is marked and is not +// an error; the interrupt does not open a new turn (it once rendered +// as a fake "u: [Request interrupted by user for tool use]" turn). +func TestAnalyzeCountsInterruptsAndDenials(t *testing.T) { + now := time.Now() + session := &parser.Session{ + ID: "human", StartTime: now, EndTime: now.Add(10 * time.Minute), + RootMessages: []*parser.Message{ + {UUID: "u1", Kind: parser.KindUserPrompt, Type: "user", Timestamp: now, + Content: []parser.ContentBlock{{Type: "text", Text: "clean up the disk"}}}, + {UUID: "a1", Kind: parser.KindAssistant, Type: "assistant", Timestamp: now.Add(time.Minute), + Content: []parser.ContentBlock{ + {Type: "text", Text: "Deleting the caches."}, + {Type: "tool_use", ToolName: "Bash", ToolID: "t1", ToolInput: map[string]any{"command": "rm -rf ~/Library/Caches"}}, + }}, + {UUID: "r1", Kind: parser.KindToolResult, Type: "user", Timestamp: now.Add(2 * time.Minute), + Content: []parser.ContentBlock{{Type: "tool_result", ToolID: "t1", IsError: true, + ToolResult: "The user doesn't want to proceed with this tool use. The tool use was rejected."}}}, + {UUID: "a2", Kind: parser.KindAssistant, Type: "assistant", Timestamp: now.Add(3 * time.Minute), + Content: []parser.ContentBlock{ + {Type: "text", Text: "Understood, listing instead."}, + {Type: "tool_use", ToolName: "Bash", ToolID: "t2", ToolInput: map[string]any{"command": "du -sh ~/Library/*"}}, + }}, + {UUID: "r2", Kind: parser.KindToolResult, Type: "user", Timestamp: now.Add(4 * time.Minute), + Content: []parser.ContentBlock{{Type: "tool_result", ToolID: "t2", IsError: true, + ToolResult: []any{map[string]any{"type": "text", "text": "Exit code 137\n[Request interrupted by user for tool use]"}}}}}, + {UUID: "i1", Kind: parser.KindInterrupt, Type: "user", Timestamp: now.Add(4*time.Minute + time.Second), + Content: []parser.ContentBlock{{Type: "text", Text: "[Request interrupted by user for tool use]"}}}, + {UUID: "u2", Kind: parser.KindUserPrompt, Type: "user", Timestamp: now.Add(5 * time.Minute), + Content: []parser.ContentBlock{{Type: "text", Text: "just report sizes, do not delete"}}}, + }, + } + result := Analyze(session) + if result.Stats.TurnCount != 2 { + t.Fatalf("turns: got %d, want 2 (interrupt must not open a turn)", result.Stats.TurnCount) + } + turn := result.Turns[0] + if turn.Interrupts != 1 || turn.Denials != 1 { + t.Fatalf("turn interventions: interrupts=%d denials=%d", turn.Interrupts, turn.Denials) + } + // The denial is not an error; the killed command (exit 137) is. + if turn.Errors != 1 { + t.Fatalf("errors: got %d, want 1", turn.Errors) + } + if len(turn.Steps) != 2 || turn.Steps[0].Denials != 1 || turn.Steps[1].Interrupts != 1 { + t.Fatalf("step attribution: %+v", turn.Steps) + } + var denied *ToolCallEvidence + for i := range turn.Steps[0].Mutations { + if turn.Steps[0].Mutations[i].ToolID == "t1" { + denied = &turn.Steps[0].Mutations[i] + } + } + if denied == nil || !denied.Denied || denied.IsError { + t.Fatalf("denied call evidence: %+v", denied) + } + if result.Stats.Interrupts != 1 || result.Stats.Denials != 1 { + t.Fatalf("stats: %+v", result.Stats) + } + outline := BuildOutline(result, 80) + if outline.Turns[0].Interrupts != 1 || outline.Turns[0].Denials != 1 || outline.Turns[0].Steps[0].Denials != 1 { + t.Fatalf("outline: %+v", outline.Turns[0]) + } + text := RenderOutlineText(outline) + if !strings.Contains(text, "1 interrupt, 1 denied") || !strings.Contains(text, "1 denied)") { + t.Fatalf("outline text must badge interventions:\n%s", text) + } +} + +// A step with tools but no narration must still get a readable +// headline in the outline (it rendered as a bare "[4t]" badge row). +func TestOutlineLabelsToolOnlySteps(t *testing.T) { + now := time.Now() + session := &parser.Session{ + ID: "tools-only", StartTime: now, EndTime: now.Add(time.Minute), + RootMessages: []*parser.Message{ + {UUID: "u1", Kind: parser.KindUserPrompt, Type: "user", Timestamp: now, + Content: []parser.ContentBlock{{Type: "text", Text: "try open chrome"}}}, + {UUID: "a1", Kind: parser.KindAssistant, Type: "assistant", Timestamp: now.Add(time.Second), + Content: []parser.ContentBlock{ + {Type: "tool_use", ToolName: "Bash", ToolID: "t1", ToolInput: map[string]any{"command": "open -a chrome"}}, + {Type: "tool_use", ToolName: "Bash", ToolID: "t2", ToolInput: map[string]any{"command": "which chrome"}}, + {Type: "tool_use", ToolName: "Read", ToolID: "t3", ToolInput: map[string]any{"file_path": "/w/x"}}, + {Type: "tool_use", ToolName: "Glob", ToolID: "t4", ToolInput: map[string]any{"pattern": "*"}}, + {Type: "tool_use", ToolName: "Grep", ToolID: "t5", ToolInput: map[string]any{"pattern": "x"}}, + }}, + }, + } + outline := BuildOutline(Analyze(session), 80) + got := outline.Turns[0].Steps[0].Headline + if got != "(no narration) Bash x2, Glob, Grep, +1 more" { + t.Fatalf("tool-only headline: %q", got) + } +} diff --git a/internal/trace/outline.go b/internal/trace/outline.go index 5c271fb..f37b355 100644 --- a/internal/trace/outline.go +++ b/internal/trace/outline.go @@ -3,6 +3,7 @@ package trace import ( "fmt" "path/filepath" + "sort" "strings" "time" ) @@ -39,6 +40,8 @@ func BuildOutline(result *TraceResult, width int) *Outline { SupersededByTurn: turn.SupersededByTurn, Edits: len(turn.FilesEdited), Errors: turn.Errors, + Interrupts: turn.Interrupts, + Denials: turn.Denials, InputTokens: turn.InputTokens, OutputTokens: turn.OutputTokens, CacheReadTokens: turn.CacheReadTokens, @@ -57,15 +60,22 @@ func BuildOutline(result *TraceResult, width int) *Outline { } for _, step := range turn.Steps { os := OutlineStep{ - Index: step.Index, - Headline: headline(step.Narration, width), - Edits: len(step.FilesEdited), - Errors: step.Errors, - Agents: len(step.Sidechains), + Index: step.Index, + Headline: headline(step.Narration, width), + Edits: len(step.FilesEdited), + Errors: step.Errors, + Interrupts: step.Interrupts, + Denials: step.Denials, + Agents: len(step.Sidechains), } for _, n := range step.ToolCounts { os.Tools += n } + if os.Headline == "" { + // A step the agent never narrated (straight to tools) + // still needs a readable line: say what it ran. + os.Headline = toolsHeadline(step) + } ot.Agents += len(step.Sidechains) ot.Steps = append(ot.Steps, os) } @@ -74,6 +84,42 @@ func BuildOutline(result *TraceResult, width int) *Outline { return outline } +// toolsHeadline labels a narration-less step from its tool sequence: +// "(no narration) Bash x3, Read" — busiest tools first, capped at +// three names, so the outline never shows a bare badge row. +func toolsHeadline(step Step) string { + if len(step.ToolCounts) == 0 { + return "(no narration)" + } + type tc struct { + name string + n int + } + var list []tc + for name, n := range step.ToolCounts { + list = append(list, tc{name, n}) + } + sort.Slice(list, func(i, j int) bool { + if list[i].n != list[j].n { + return list[i].n > list[j].n + } + return list[i].name < list[j].name + }) + var parts []string + for i, t := range list { + if i == 3 { + parts = append(parts, fmt.Sprintf("+%d more", len(list)-3)) + break + } + if t.n > 1 { + parts = append(parts, fmt.Sprintf("%s x%d", t.name, t.n)) + } else { + parts = append(parts, t.name) + } + } + return "(no narration) " + strings.Join(parts, ", ") +} + // headline reduces evidence text to its first meaningful line, capped // at width runes for outline display (<= 0 = uncapped). func headline(text string, width int) string { @@ -135,11 +181,25 @@ func RenderOutlineText(outline *Outline) string { // Wall-span misleads on long sessions; print active time next to it // and say which timezone the rendered times are in (JSON carries // them in UTC — silent localization breaks cross-referencing). - fmt.Fprintf(&b, "%s -> %s (times %s) | active %s | %s | %d steps | %d files edited | %d tool errors%s\n", + // Human interventions belong in the header: a session with 4 + // interrupts and 2 denials reads very differently from a clean run. + humanSeg := "" + if outline.Stats.Interrupts > 0 || outline.Stats.Denials > 0 { + var parts []string + if outline.Stats.Interrupts > 0 { + parts = append(parts, plural(outline.Stats.Interrupts, "interrupt")) + } + if outline.Stats.Denials > 0 { + parts = append(parts, fmt.Sprintf("%d denied", outline.Stats.Denials)) + } + humanSeg = " | " + strings.Join(parts, ", ") + } + fmt.Fprintf(&b, "%s -> %s (times %s) | active %s | %s | %d steps | %d files edited | %d tool errors%s%s\n", formatOutlineTime(s.Start), formatOutlineTime(s.End), outlineZone(s.Start), formatActive(outline.Stats.ActiveSecs), turnsSeg, outline.Stats.StepCount, outline.Stats.FilesEdited, outline.Stats.ToolErrors, + humanSeg, headerCost(outline.Stats.TotalCostUSD, outline.Stats.AgentsCostUSD)) if tok := tokenSplit(outline.Stats.InputTokens, outline.Stats.OutputTokens, outline.Stats.CacheReadTokens, outline.Stats.CacheCreateTokens, @@ -193,6 +253,12 @@ func turnBadges(turn OutlineTurn) string { if turn.Errors > 0 { parts = append(parts, fmt.Sprintf("%d errors", turn.Errors)) } + if turn.Interrupts > 0 { + parts = append(parts, plural(turn.Interrupts, "interrupt")) + } + if turn.Denials > 0 { + parts = append(parts, fmt.Sprintf("%d denied", turn.Denials)) + } if turn.Agents > 0 { parts = append(parts, fmt.Sprintf("%d agents", turn.Agents)) } @@ -261,6 +327,12 @@ func stepBadges(step OutlineStep) string { if step.Errors > 0 { parts = append(parts, fmt.Sprintf("%dx", step.Errors)) } + if step.Interrupts > 0 { + parts = append(parts, fmt.Sprintf("%d!", step.Interrupts)) + } + if step.Denials > 0 { + parts = append(parts, fmt.Sprintf("%dd", step.Denials)) + } if step.Agents > 0 { parts = append(parts, fmt.Sprintf("%da", step.Agents)) } @@ -270,6 +342,13 @@ func stepBadges(step OutlineStep) string { return " [" + strings.Join(parts, " ") + "]" } +func plural(n int, word string) string { + if n == 1 { + return fmt.Sprintf("1 %s", word) + } + return fmt.Sprintf("%d %ss", n, word) +} + func commitBadge(shas []string) string { if len(shas) == 0 { return "" diff --git a/internal/trace/related.go b/internal/trace/related.go new file mode 100644 index 0000000..e99d37d --- /dev/null +++ b/internal/trace/related.go @@ -0,0 +1,485 @@ +package trace + +import ( + "path/filepath" + "regexp" + "sort" + "strings" + "time" + "unicode/utf8" + + "github.com/thevibeworks/ccx/internal/parser" +) + +// Session connections (docs/design/0006-session-connections.md): the +// deterministic relations between one anchor session and the other +// sessions of its workspace, each backed by evidence a reader can walk +// to. Stated from the anchor's point of view: "anchor forked_from X", +// "anchor handoff_from X" (anchor read what X wrote). + +// RelatedSession is one session connected to the anchor. +type RelatedSession struct { + SessionID string `json:"session_id"` + Provider string `json:"provider,omitempty"` + Summary string `json:"summary,omitempty"` + Start time.Time `json:"start"` + End time.Time `json:"end"` + Strength string `json:"strength"` // strong | medium | weak + Relations []Relation `json:"relations"` +} + +// Relation is one kind of link plus its evidence. +type Relation struct { + Kind string `json:"kind"` + // Count is the total behind a sampled list: shared message uuids + // for forks, files for builds_on. Paths/Evidence are capped; + // Truncated says so. + Count int `json:"count,omitempty"` + Paths []string `json:"paths,omitempty"` + Evidence []RelationEvidence `json:"evidence,omitempty"` + Truncated bool `json:"truncated,omitempty"` +} + +// RelationEvidence is one anchor into a transcript: which session, +// which message, when, and (for file relations) which path. +type RelationEvidence struct { + SessionID string `json:"session_id"` + MessageID string `json:"message_id,omitempty"` + Time time.Time `json:"time,omitempty"` + Path string `json:"path,omitempty"` + Quote string `json:"quote,omitempty"` +} + +const ( + RelForkedFrom = "forked_from" + RelForkOf = "fork_of" + RelMentions = "mentions" + RelMentionedBy = "mentioned_by" + RelHandoffFrom = "handoff_from" + RelHandoffTo = "handoff_to" + RelBuildsOn = "builds_on" + RelBuiltOnBy = "built_on_by" + RelOverlaps = "overlaps" + RelPrevious = "previous" + RelNext = "next" + + StrengthStrong = "strong" + StrengthMedium = "medium" + StrengthWeak = "weak" + + // maxRelationPaths bounds the file list carried per relation; the + // count says how many there were. + maxRelationPaths = 5 + // idRefLen is how much of a session id must appear in text to + // count as a mention: 8 hex chars, the prefix ccx prints everywhere. + idRefLen = 8 +) + +// relationStrength is the one shared band table (0005 principle 7). +var relationStrength = map[string]string{ + RelForkedFrom: StrengthStrong, + RelForkOf: StrengthStrong, + RelMentions: StrengthStrong, + RelMentionedBy: StrengthStrong, + RelHandoffFrom: StrengthStrong, + RelHandoffTo: StrengthStrong, + RelBuildsOn: StrengthMedium, + RelBuiltOnBy: StrengthMedium, + RelOverlaps: StrengthMedium, + RelPrevious: StrengthWeak, + RelNext: StrengthWeak, +} + +var strengthRank = map[string]int{StrengthStrong: 0, StrengthMedium: 1, StrengthWeak: 2} + +// SessionProfile is what relation detection needs from one parsed +// session; building it is the only per-session cost, so callers can +// profile every session of a workspace once and relate any pair. +type SessionProfile struct { + ID string + Provider string + Summary string + Start time.Time + End time.Time + + uuids map[string]struct{} + edits map[string][]touch // path -> edits, chronological + touches map[string][]touch // path -> reads and edits, chronological + idRefs map[string]refHit // lowercase 8-hex prefix -> first message quoting it +} + +type touch struct { + msgID string + t time.Time +} + +type refHit struct { + msgID string + t time.Time + quote string +} + +var hexRunRe = regexp.MustCompile(`[0-9a-fA-F]{8,}`) + +// ProfileSession extracts the relation signals from one parsed session. +func ProfileSession(s *parser.Session) *SessionProfile { + p := &SessionProfile{ + uuids: make(map[string]struct{}), + edits: make(map[string][]touch), + touches: make(map[string][]touch), + idRefs: make(map[string]refHit), + } + if s == nil { + return p + } + p.ID = s.ID + p.Provider = s.Provider + p.Summary = s.Summary + p.Start = s.StartTime + p.End = s.EndTime + + for _, msg := range parser.FlattenSessionMessages(s) { + if msg.UUID != "" { + p.uuids[msg.UUID] = struct{}{} + } + if msg.Kind == parser.KindUserPrompt || msg.Kind == parser.KindAssistant { + for _, b := range msg.Content { + if b.Type != "text" && b.Type != "thinking" { + continue + } + p.collectIDRefs(msg, b.Text) + } + } + for _, cb := range msg.Content { + if cb.Type != "tool_use" || cb.ToolName == "" { + continue + } + paths := extractPaths(cb.ToolInput) + if len(paths) == 0 { + continue + } + isEdit := mutatingTools[cb.ToolName] + isRead := readTools[cb.ToolName] + if !isEdit && !isRead { + continue + } + for _, path := range paths { + path = absolutePath(path, s.CWD) + if path == "" { + continue + } + t := touch{msgID: msg.UUID, t: msg.Timestamp} + p.touches[path] = append(p.touches[path], t) + if isEdit { + p.edits[path] = append(p.edits[path], t) + } + } + } + } + for _, m := range []map[string][]touch{p.edits, p.touches} { + for path := range m { + list := m[path] + sort.SliceStable(list, func(i, j int) bool { return list[i].t.Before(list[j].t) }) + m[path] = list + } + } + return p +} + +func (p *SessionProfile) collectIDRefs(msg *parser.Message, text string) { + for _, loc := range hexRunRe.FindAllStringIndex(text, -1) { + prefix := strings.ToLower(text[loc[0] : loc[0]+idRefLen]) + if _, seen := p.idRefs[prefix]; seen { + continue + } + p.idRefs[prefix] = refHit{ + msgID: msg.UUID, + t: msg.Timestamp, + quote: quoteAround(text, loc[0], loc[1]-loc[0]), + } + } +} + +// absolutePath makes tool paths comparable across sessions of one +// workspace: relative paths (Codex, shell redirects) are joined onto +// the session cwd. +func absolutePath(path, cwd string) string { + path = cleanEvidencePath(path) + if path == "" { + return "" + } + if !filepath.IsAbs(path) && cwd != "" { + path = filepath.Join(cwd, path) + } + return filepath.Clean(path) +} + +// isBatonPath reports whether a path is a session baton — the files +// one session writes so the next can pick the work up. +func isBatonPath(path string) bool { + lower := strings.ToLower(filepath.ToSlash(path)) + base := filepath.Base(lower) + switch { + case strings.HasPrefix(base, "handoff") && (strings.HasSuffix(base, ".md") || strings.HasSuffix(base, ".org")): + return true + case strings.Contains(lower, "/handoffs/"): + return true + case strings.Contains(lower, "devlog"): + return true + case base == "plan.md", base == "todo.md", base == "notes.md": + return true + } + return false +} + +// idPrefix is the lowercase 8-hex prefix ccx prints for a session id; +// empty when the id does not start with one. +func idPrefix(id string) string { + id = strings.ToLower(id) + if len(id) < idRefLen { + return "" + } + for i := 0; i < idRefLen; i++ { + c := id[i] + if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f')) { + return "" + } + } + return id[:idRefLen] +} + +// RelateSessions computes the anchor's connections to others (any +// order; the anchor itself is skipped if present). Result is strongest +// first, then by start time. +func RelateSessions(anchor *SessionProfile, others []*SessionProfile) []RelatedSession { + if anchor == nil { + return nil + } + var out []RelatedSession + var prev, next *SessionProfile + for _, o := range others { + if o == nil || o.ID == anchor.ID { + continue + } + if !o.Start.IsZero() && !anchor.Start.IsZero() { + if o.Start.Before(anchor.Start) && (prev == nil || o.Start.After(prev.Start)) { + prev = o + } + if o.Start.After(anchor.Start) && (next == nil || o.Start.Before(next.Start)) { + next = o + } + } + } + for _, o := range others { + if o == nil || o.ID == anchor.ID { + continue + } + rels := relatePair(anchor, o) + if o == prev { + rels = append(rels, Relation{Kind: RelPrevious}) + } + if o == next { + rels = append(rels, Relation{Kind: RelNext}) + } + if len(rels) == 0 { + continue + } + out = append(out, RelatedSession{ + SessionID: o.ID, + Provider: o.Provider, + Summary: o.Summary, + Start: o.Start, + End: o.End, + Strength: strongest(rels), + Relations: rels, + }) + } + sort.SliceStable(out, func(i, j int) bool { + a, b := out[i], out[j] + if strengthRank[a.Strength] != strengthRank[b.Strength] { + return strengthRank[a.Strength] < strengthRank[b.Strength] + } + return a.Start.Before(b.Start) + }) + return out +} + +func strongest(rels []Relation) string { + best := StrengthWeak + for _, r := range rels { + if s := relationStrength[r.Kind]; strengthRank[s] < strengthRank[best] { + best = s + } + } + return best +} + +// relatePair finds every relation between anchor a and other o, from +// a's point of view. +func relatePair(a, o *SessionProfile) []Relation { + var rels []Relation + + // Fork: shared message uuids; the earlier session is the origin. + if shared, first := sharedUUIDs(a, o); shared > 0 { + kind := RelForkOf + if !a.Start.IsZero() && !o.Start.IsZero() && a.Start.After(o.Start) { + kind = RelForkedFrom + } + rels = append(rels, Relation{ + Kind: kind, + Count: shared, + Evidence: []RelationEvidence{{SessionID: a.ID, MessageID: first}, {SessionID: o.ID, MessageID: first}}, + }) + } + + // Mentions: one session's text names the other's id. + if pfx := idPrefix(o.ID); pfx != "" { + if hit, ok := a.idRefs[pfx]; ok { + rels = append(rels, Relation{Kind: RelMentions, Evidence: []RelationEvidence{ + {SessionID: a.ID, MessageID: hit.msgID, Time: hit.t, Quote: hit.quote}, + }}) + } + } + if pfx := idPrefix(a.ID); pfx != "" { + if hit, ok := o.idRefs[pfx]; ok { + rels = append(rels, Relation{Kind: RelMentionedBy, Evidence: []RelationEvidence{ + {SessionID: o.ID, MessageID: hit.msgID, Time: hit.t, Quote: hit.quote}, + }}) + } + } + + // Files: o wrote, a touched later -> a handoff_from / builds_on o; + // a wrote, o touched later -> a handoff_to / built_on_by o. + if h, b := fileLinks(o, a); h != nil || b != nil { + if h != nil { + h.Kind = RelHandoffFrom + rels = append(rels, *h) + } + if b != nil { + b.Kind = RelBuildsOn + rels = append(rels, *b) + } + } + if h, b := fileLinks(a, o); h != nil || b != nil { + if h != nil { + h.Kind = RelHandoffTo + rels = append(rels, *h) + } + if b != nil { + b.Kind = RelBuiltOnBy + rels = append(rels, *b) + } + } + + // Overlap: concurrent windows. + if !a.Start.IsZero() && !a.End.IsZero() && !o.Start.IsZero() && !o.End.IsZero() && + a.Start.Before(o.End) && o.Start.Before(a.End) { + start, end := a.Start, a.End + if o.Start.After(start) { + start = o.Start + } + if o.End.Before(end) { + end = o.End + } + rels = append(rels, Relation{Kind: RelOverlaps, Evidence: []RelationEvidence{ + {SessionID: a.ID, Time: start}, {SessionID: o.ID, Time: end}, + }}) + } + return rels +} + +func sharedUUIDs(a, o *SessionProfile) (int, string) { + small, large := a.uuids, o.uuids + if len(large) < len(small) { + small, large = large, small + } + n := 0 + first := "" + for id := range small { + if _, ok := large[id]; ok { + n++ + if first == "" || id < first { + first = id + } + } + } + return n, first +} + +// fileLinks finds paths writer edited that reader touched afterwards, +// split into baton files (handoff) and everything else (builds_on). +// Evidence pairs the writer's first qualifying edit with the reader's +// first touch after it. Paths are sorted for deterministic output. +func fileLinks(writer, reader *SessionProfile) (handoff, builds *Relation) { + paths := make([]string, 0, len(writer.edits)) + for path := range writer.edits { + if _, ok := reader.touches[path]; ok { + paths = append(paths, path) + } + } + sort.Strings(paths) + + add := func(rel **Relation, path string, w, r touch) { + if *rel == nil { + *rel = &Relation{} + } + (*rel).Count++ + if len((*rel).Paths) >= maxRelationPaths { + (*rel).Truncated = true + return + } + (*rel).Paths = append((*rel).Paths, path) + (*rel).Evidence = append((*rel).Evidence, + RelationEvidence{SessionID: writer.ID, MessageID: w.msgID, Time: w.t, Path: path}, + RelationEvidence{SessionID: reader.ID, MessageID: r.msgID, Time: r.t, Path: path}, + ) + } + + for _, path := range paths { + w := writer.edits[path][0] + var r touch + found := false + for _, t := range reader.touches[path] { + if t.t.After(w.t) { + r, found = t, true + break + } + } + if !found { + continue + } + if isBatonPath(path) { + add(&handoff, path, w, r) + } else { + add(&builds, path, w, r) + } + } + return handoff, builds +} + +// quoteAround cuts a bounded window around a match, clamped to rune +// boundaries, for evidence quotes. +func quoteAround(text string, idx, n int) string { + start := idx - 40 + if start < 0 { + start = 0 + } + end := idx + n + 40 + if end > len(text) { + end = len(text) + } + for start > 0 && !utf8.RuneStart(text[start]) { + start-- + } + for end < len(text) && !utf8.RuneStart(text[end]) { + end++ + } + out := strings.Join(strings.Fields(text[start:end]), " ") + if start > 0 { + out = "..." + out + } + if end < len(text) { + out += "..." + } + return out +} diff --git a/internal/trace/related_test.go b/internal/trace/related_test.go new file mode 100644 index 0000000..7607ecc --- /dev/null +++ b/internal/trace/related_test.go @@ -0,0 +1,227 @@ +package trace + +import ( + "testing" + "time" + + "github.com/thevibeworks/ccx/internal/parser" +) + +func relT(min int) time.Time { return time.Date(2026, 8, 18, 10, min, 0, 0, time.UTC) } + +func relSession(id string, start, end time.Time, msgs ...*parser.Message) *parser.Session { + return &parser.Session{ID: id, Provider: "claude-code", CWD: "/w", StartTime: start, EndTime: end, RootMessages: msgs} +} + +func relUser(uuid string, t time.Time, text string) *parser.Message { + return &parser.Message{UUID: uuid, Type: "user", Kind: parser.KindUserPrompt, Timestamp: t, + Content: []parser.ContentBlock{{Type: "text", Text: text}}} +} + +func relTool(uuid string, t time.Time, tool string, input map[string]any) *parser.Message { + return &parser.Message{UUID: uuid, Type: "assistant", Kind: parser.KindAssistant, Timestamp: t, + Content: []parser.ContentBlock{{Type: "tool_use", ToolName: tool, ToolID: "t-" + uuid, ToolInput: input}}} +} + +func kinds(rels []Relation) map[string]Relation { + m := make(map[string]Relation) + for _, r := range rels { + m[r.Kind] = r + } + return m +} + +func findRelated(list []RelatedSession, id string) *RelatedSession { + for i := range list { + if list[i].SessionID == id { + return &list[i] + } + } + return nil +} + +// A handoff is a baton file written by one session and read by a +// later one; an ordinary file edited then read is builds_on; both +// carry writer and reader anchors. Direction flips with the point of +// view, and a read that happened BEFORE the write is not a link. +func TestRelateSessionsHandoffAndBuildsOn(t *testing.T) { + writer := relSession("aaaaaaaa-1", relT(0), relT(10), + relTool("w1", relT(1), "Write", map[string]any{"file_path": "/w/HANDOFF.md"}), + relTool("w2", relT(2), "Edit", map[string]any{"file_path": "/w/src/main.go"}), + relTool("w3", relT(3), "Read", map[string]any{"file_path": "/w/README.md"}), + ) + reader := relSession("bbbbbbbb-2", relT(20), relT(30), + relTool("r1", relT(21), "Read", map[string]any{"file_path": "/w/HANDOFF.md"}), + relTool("r2", relT(22), "Read", map[string]any{"file_path": "src/main.go"}), // relative: joined onto cwd + relTool("r3", relT(23), "Edit", map[string]any{"file_path": "/w/README.md"}), + ) + pw, pr := ProfileSession(writer), ProfileSession(reader) + + got := RelateSessions(pr, []*SessionProfile{pw}) + if len(got) != 1 || got[0].SessionID != "aaaaaaaa-1" || got[0].Strength != StrengthStrong { + t.Fatalf("reader view: %+v", got) + } + k := kinds(got[0].Relations) + h, ok := k[RelHandoffFrom] + if !ok || h.Count != 1 || h.Paths[0] != "/w/HANDOFF.md" || len(h.Evidence) != 2 { + t.Fatalf("handoff_from: %+v", h) + } + if h.Evidence[0].SessionID != "aaaaaaaa-1" || h.Evidence[0].MessageID != "w1" || h.Evidence[1].MessageID != "r1" { + t.Fatalf("handoff evidence must pair writer then reader: %+v", h.Evidence) + } + b, ok := k[RelBuildsOn] + if !ok || b.Count != 1 || b.Paths[0] != "/w/src/main.go" { + t.Fatalf("builds_on: %+v (README was read by writer, not edited — must not count)", b) + } + if _, ok := k[RelBuiltOnBy]; ok { + t.Fatal("reader edited README after writer READ it; that is not built_on_by") + } + if _, ok := k[RelPrevious]; !ok { + t.Fatal("writer is the reader's nearest earlier session") + } + + got = RelateSessions(pw, []*SessionProfile{pr}) + k = kinds(got[0].Relations) + if _, ok := k[RelHandoffTo]; !ok { + t.Fatalf("writer view must say handoff_to: %+v", k) + } + if _, ok := k[RelBuiltOnBy]; !ok { + t.Fatalf("writer view must say built_on_by: %+v", k) + } + if _, ok := k[RelNext]; !ok { + t.Fatal("reader is the writer's next session") + } +} + +// Shared message uuids mean a fork; the earlier session is the origin. +// A mention is the other session's 8-hex prefix in conversation text +// (not in tool results), quoted as evidence. +func TestRelateSessionsForkAndMentions(t *testing.T) { + origin := relSession("11111111-aaaa", relT(0), relT(5), + relUser("shared-1", relT(0), "start"), + relUser("shared-2", relT(1), "more"), + ) + fork := relSession("22222222-bbbb", relT(10), relT(15), + relUser("shared-1", relT(0), "start"), + relUser("shared-2", relT(1), "more"), + relUser("own-3", relT(11), "continue from session 11111111 please"), + ) + // A tool result naming the id is not a mention. + bystander := relSession("33333333-cccc", relT(20), relT(25), + &parser.Message{UUID: "tr", Type: "user", Kind: parser.KindToolResult, Timestamp: relT(21), + Content: []parser.ContentBlock{{Type: "tool_result", ToolResult: "22222222-bbbb"}, {Type: "text", Text: "session 22222222 listed"}}}, + ) + po, pf, pb := ProfileSession(origin), ProfileSession(fork), ProfileSession(bystander) + + got := RelateSessions(pf, []*SessionProfile{po, pb}) + r := findRelated(got, "11111111-aaaa") + if r == nil || r.Strength != StrengthStrong { + t.Fatalf("fork view of origin: %+v", got) + } + k := kinds(r.Relations) + f, ok := k[RelForkedFrom] + if !ok || f.Count != 2 || f.Evidence[0].MessageID != "shared-1" { + t.Fatalf("forked_from: %+v", f) + } + m, ok := k[RelMentions] + if !ok || m.Evidence[0].MessageID != "own-3" || m.Evidence[0].SessionID != "22222222-bbbb" || m.Evidence[0].Quote == "" { + t.Fatalf("mentions: %+v", m) + } + if b := findRelated(got, "33333333-cccc"); b != nil { + for _, rel := range b.Relations { + if rel.Kind == RelMentionedBy { + t.Fatalf("tool-result text must not count as a mention: %+v", rel) + } + } + } + + got = RelateSessions(po, []*SessionProfile{pf}) + k = kinds(got[0].Relations) + if _, ok := k[RelForkOf]; !ok { + t.Fatalf("origin view must say fork_of: %+v", k) + } + if _, ok := k[RelMentionedBy]; !ok { + t.Fatalf("origin view must say mentioned_by: %+v", k) + } +} + +// Overlap needs intersecting windows; ordering is strongest first, +// then by start; unrelated sessions are absent; the anchor is skipped. +func TestRelateSessionsOverlapOrderingAndSelf(t *testing.T) { + anchor := relSession("aaaaaaaa-0", relT(10), relT(20), + relTool("a1", relT(12), "Edit", map[string]any{"file_path": "/w/x.go"})) + concurrent := relSession("bbbbbbbb-1", relT(15), relT(25)) + later := relSession("cccccccc-2", relT(30), relT(35), + relTool("c1", relT(31), "Read", map[string]any{"file_path": "/w/x.go"})) + unrelated := relSession("dddddddd-3", relT(40), relT(45), + relTool("d1", relT(41), "Read", map[string]any{"file_path": "/w/other.go"})) + pa := ProfileSession(anchor) + got := RelateSessions(pa, []*SessionProfile{ProfileSession(unrelated), ProfileSession(later), ProfileSession(concurrent), pa}) + + if len(got) != 2 { + t.Fatalf("want concurrent + later only, got %+v", got) + } + // Both medium: order by start. + if got[0].SessionID != "bbbbbbbb-1" || got[1].SessionID != "cccccccc-2" { + t.Fatalf("order: %s, %s", got[0].SessionID, got[1].SessionID) + } + k := kinds(got[0].Relations) + o, ok := k[RelOverlaps] + if !ok || !o.Evidence[0].Time.Equal(relT(15)) || !o.Evidence[1].Time.Equal(relT(20)) { + t.Fatalf("overlap window: %+v", o) + } + if _, ok := kinds(got[1].Relations)[RelBuiltOnBy]; !ok { + t.Fatalf("later read of anchor's edit: %+v", got[1].Relations) + } + // Timeless sessions never become previous/next. + if got := RelateSessions(pa, []*SessionProfile{ProfileSession(relSession("eeeeeeee-4", time.Time{}, time.Time{}))}); len(got) != 0 { + t.Fatalf("timeless session related: %+v", got) + } +} + +// Path lists are capped at maxRelationPaths with the count intact and +// Truncated set — never a silent cut. +func TestRelateSessionsCapsPaths(t *testing.T) { + var wmsgs, rmsgs []*parser.Message + for i := 0; i < maxRelationPaths+3; i++ { + p := "/w/f" + string(rune('a'+i)) + ".go" + wmsgs = append(wmsgs, relTool("w"+string(rune('a'+i)), relT(1), "Edit", map[string]any{"file_path": p})) + rmsgs = append(rmsgs, relTool("r"+string(rune('a'+i)), relT(30), "Read", map[string]any{"file_path": p})) + } + pw := ProfileSession(relSession("aaaaaaaa-w", relT(0), relT(5), wmsgs...)) + pr := ProfileSession(relSession("bbbbbbbb-r", relT(20), relT(40), rmsgs...)) + got := RelateSessions(pr, []*SessionProfile{pw}) + b := kinds(got[0].Relations)[RelBuildsOn] + if b.Count != maxRelationPaths+3 || len(b.Paths) != maxRelationPaths || !b.Truncated { + t.Fatalf("cap: count=%d paths=%d truncated=%v", b.Count, len(b.Paths), b.Truncated) + } +} + +func TestIsBatonPath(t *testing.T) { + yes := []string{"/w/HANDOFF.md", "/w/handoff-notes.org", "/w/.claude/handoffs/latest.md", "/w/docs/devlog/2026-08-18-x.org", "/w/DEVLOG.org", "/w/PLAN.md", "/w/todo.md"} + no := []string{"/w/README.md", "/w/internal/cmd/search.go", "/w/handoff.go", "/w/docs/design/0006.md"} + for _, p := range yes { + if !isBatonPath(p) { + t.Errorf("%s should be a baton path", p) + } + } + for _, p := range no { + if isBatonPath(p) { + t.Errorf("%s should not be a baton path", p) + } + } +} + +func TestIDPrefix(t *testing.T) { + cases := map[string]string{ + "736a7bac-0a5d-4e3f-9036-c8a94111a347": "736a7bac", + "019F6528-ABCD": "019f6528", + "short": "", + "zzzzzzzz-not-hex": "", + } + for in, want := range cases { + if got := idPrefix(in); got != want { + t.Errorf("idPrefix(%q) = %q, want %q", in, got, want) + } + } +} diff --git a/internal/trace/related_workspace.go b/internal/trace/related_workspace.go new file mode 100644 index 0000000..7ddd0f6 --- /dev/null +++ b/internal/trace/related_workspace.go @@ -0,0 +1,104 @@ +package trace + +import ( + "fmt" + "path/filepath" + "strings" + "sync" + + "github.com/thevibeworks/ccx/internal/catalog" + "github.com/thevibeworks/ccx/internal/parser" +) + +// SessionSource is the slice of provider.Backend that workspace +// relation needs; narrowed so the CLI, the web API, and tests share +// one implementation. +type SessionSource interface { + ListSessions(query catalog.SessionQuery) ([]*parser.Session, error) + ParseSession(filePath string) (*parser.Session, error) +} + +// RelateWorkspace profiles every session of the anchor's workspace +// (all providers) on `workers` goroutines and returns the anchor's +// connections. tick, when non-nil, is called once per session +// profiled (progress). Sessions that fail to parse become warnings, +// never silent gaps. +func RelateWorkspace(src SessionSource, anchor *parser.Session, workers int, tick func()) ([]RelatedSession, []TraceWarning, error) { + if src == nil || anchor == nil { + return nil, nil, fmt.Errorf("no session") + } + query := catalog.SessionQuery{Scope: catalog.ScopeProject, ProjectName: anchor.ProjectName} + if strings.TrimSpace(anchor.CWD) != "" { + query = catalog.SessionQuery{Scope: catalog.ScopeWorkspace, WorkspacePath: anchor.CWD} + } + sessions, err := src.ListSessions(query.WithoutLimit().WithoutProviderFilter()) + if err != nil { + return nil, nil, fmt.Errorf("list workspace sessions: %w", err) + } + // The anchor may be missing from the workspace listing when its + // cwd differs from the project path (a fork into another dir); + // it always takes part. + found := false + for _, s := range sessions { + if s.FilePath == anchor.FilePath { + found = true + break + } + } + if !found { + sessions = append(sessions, anchor) + } + if workers < 1 { + workers = 1 + } + if workers > len(sessions) { + workers = len(sessions) + } + + profiles := make([]*SessionProfile, len(sessions)) + var warnings []TraceWarning + var warnMu sync.Mutex + var wg sync.WaitGroup + next := make(chan int) + for w := 0; w < workers; w++ { + wg.Add(1) + go func() { + defer wg.Done() + for i := range next { + full, err := src.ParseSession(sessions[i].FilePath) + if err != nil { + warnMu.Lock() + warnings = append(warnings, TraceWarning{Kind: "related_parse_failed", Message: fmt.Sprintf("skipping %s: %v", filepath.Base(sessions[i].FilePath), err)}) + warnMu.Unlock() + } else { + profiles[i] = ProfileSession(full) + } + if tick != nil { + tick() + } + } + }() + } + for i := range sessions { + next <- i + } + close(next) + wg.Wait() + + var anchorProfile *SessionProfile + others := make([]*SessionProfile, 0, len(profiles)) + for i, p := range profiles { + if p == nil { + continue + } + if sessions[i].FilePath == anchor.FilePath { + anchorProfile = p + continue + } + others = append(others, p) + } + if anchorProfile == nil { + return nil, warnings, fmt.Errorf("could not parse anchor session %s", anchor.ID) + } + return RelateSessions(anchorProfile, others), warnings, nil +} diff --git a/internal/trace/types.go b/internal/trace/types.go index 7adbe08..779af20 100644 --- a/internal/trace/types.go +++ b/internal/trace/types.go @@ -20,8 +20,13 @@ type TraceResult struct { Sidechains []Sidechain `json:"sidechains,omitempty"` Git GitCorrelation `json:"git"` Workspace WorkspaceContext `json:"workspace_context"` - Stats TraceStats `json:"stats"` - Warnings []TraceWarning `json:"warnings,omitempty"` + // Related is the session's connections to the other sessions of + // its workspace (docs/design/0006-session-connections.md); filled + // only for the full bundle, since it costs a parse of every + // session in the workspace. + Related []RelatedSession `json:"related,omitempty"` + Stats TraceStats `json:"stats"` + Warnings []TraceWarning `json:"warnings,omitempty"` } type SessionMeta struct { @@ -65,6 +70,13 @@ type Turn struct { FilesRead []string `json:"files_read,omitempty"` ToolCounts map[string]int `json:"tool_counts,omitempty"` Errors int `json:"errors,omitempty"` + // Human interventions inside the turn: Interrupts counts + // "[Request interrupted by user]" markers (the human pressed + // stop); Denials counts tool calls the human rejected at the + // permission prompt. Both are the human in the loop, distinct + // from a new prompt and from tool errors. + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` // Token split for the main loop. Cache tokens dominate real cost // (cache writes bill at 1.25x input, and one long turn can write // hundreds of thousands), so a cost without them is unauditable. @@ -106,6 +118,8 @@ type Step struct { // are summarized by ToolCounts and the turn-level file lists. Mutations []ToolCallEvidence `json:"mutations,omitempty"` Errors int `json:"errors,omitempty"` + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` Sidechains []Sidechain `json:"sidechains,omitempty"` CostUSD float64 `json:"cost_usd,omitempty"` } @@ -125,6 +139,9 @@ type ToolCallEvidence struct { MutatesWorkspace bool `json:"mutates_workspace"` Reads bool `json:"reads"` IsError bool `json:"is_error,omitempty"` + // Denied: the human rejected this call at the permission prompt. + // Such a call is not an error and did not run. + Denied bool `json:"denied,omitempty"` } type Sidechain struct { @@ -213,6 +230,8 @@ type TraceStats struct { FilesRead int `json:"files_read"` ToolsUsed int `json:"tools_used"` ToolErrors int `json:"tool_errors"` + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` WorkspaceDocs int `json:"workspace_docs"` KnowledgeEntries int `json:"knowledge_entries"` CommitsLinked int `json:"commits_linked"` @@ -266,6 +285,8 @@ type OutlineTurn struct { Edits int `json:"edits,omitempty"` Tools int `json:"tools,omitempty"` Errors int `json:"errors,omitempty"` + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` Agents int `json:"agents,omitempty"` InputTokens int `json:"input_tokens,omitempty"` OutputTokens int `json:"output_tokens,omitempty"` @@ -278,10 +299,12 @@ type OutlineTurn struct { } type OutlineStep struct { - Index int `json:"index"` - Headline string `json:"headline"` - Tools int `json:"tools,omitempty"` - Edits int `json:"edits,omitempty"` - Errors int `json:"errors,omitempty"` - Agents int `json:"agents,omitempty"` + Index int `json:"index"` + Headline string `json:"headline"` + Tools int `json:"tools,omitempty"` + Edits int `json:"edits,omitempty"` + Errors int `json:"errors,omitempty"` + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` + Agents int `json:"agents,omitempty"` } diff --git a/internal/web/server.go b/internal/web/server.go index cec3240..57328ca 100644 --- a/internal/web/server.go +++ b/internal/web/server.go @@ -100,6 +100,7 @@ func Serve(addr string, backend provider.Backend) error { mux.HandleFunc("/api/sessions", handleAPISessions) mux.HandleFunc("/api/sessions/", handleAPISessions) mux.HandleFunc("/api/session/", handleAPISession) + mux.HandleFunc("/api/related/", handleAPIRelated) mux.HandleFunc("/api/stats", handleAPIStats) mux.HandleFunc("/api/settings", handleAPISettings) mux.HandleFunc("/api/export/", handleAPIExport) @@ -789,6 +790,54 @@ func handleAPISession(w http.ResponseWriter, r *http.Request) { _ = json.NewEncoder(w).Encode(fullSession) } +// handleAPIRelated serves the anchor session's connections to the other +// sessions of its workspace (docs/design/0006-session-connections.md): +// GET /api/related// -> ccx.related.v1, the same +// envelope as `ccx related --json`. Costs a parse of every workspace +// session (cached after the first call), so callers should fetch it on +// demand, not on every page load. +func handleAPIRelated(w http.ResponseWriter, r *http.Request) { + path := strings.TrimPrefix(r.URL.Path, "/api/related/") + parts := strings.SplitN(path, "/", 2) + if len(parts) != 2 { + http.NotFound(w, r) + return + } + projectName, sessionID := parts[0], parts[1] + session, err := sessionProvider.FindSession(projectName, sessionID) + if err != nil || session == nil { + http.NotFound(w, r) + return + } + related, warnings, err := trace.RelateWorkspace(sessionProvider, session, 4, nil) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + limit := 50 + if v := r.URL.Query().Get("limit"); v != "" { + if n, err := strconv.Atoi(v); err == nil && n >= 0 { + limit = n + } + } + total := len(related) + if limit > 0 && total > limit { + related = related[:limit] + } + if related == nil { + related = []trace.RelatedSession{} + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(map[string]any{ + "kind": "ccx.related.v1", + "session": map[string]any{"id": session.ID, "provider": session.Provider, "project": session.ProjectName, "path": session.FilePath}, + "related": related, + "total": total, + "shown": len(related), + "warnings": warnings, + }) +} + func handleAPIStats(w http.ResponseWriter, r *http.Request) { projects, err := sessionProvider.DiscoverProjects() if err != nil { diff --git a/internal/web/server_test.go b/internal/web/server_test.go index ea5c36a..6ddc170 100644 --- a/internal/web/server_test.go +++ b/internal/web/server_test.go @@ -1386,3 +1386,72 @@ func TestHandleAPIFile_DeniesSymlinkEscape(t *testing.T) { t.Fatalf("handleAPIFile returned %d, want %d. Body: %s", w.Code, http.StatusForbidden, w.Body.String()) } } + +// /api/related// returns the ccx.related.v1 envelope +// with the anchor's connections; two sessions in one workspace where +// the later one reads what the earlier one wrote relate as builds_on. +func TestHandleAPIRelated(t *testing.T) { + dir := setupTestDir(t) + projectDir := filepath.Join(dir, "projects", "-test-project") + later := `{"type":"user","timestamp":"2024-01-02T10:00:00Z","uuid":"u2","sessionId":"later-456","cwd":"/test/project","message":{"content":"continue"}} +{"type":"assistant","timestamp":"2024-01-02T10:00:01Z","uuid":"a2","parentUuid":"u2","message":{"content":[{"type":"tool_use","id":"t1","name":"Read","input":{"file_path":"/test/project/main.go"}}]}} +` + earlier := `{"type":"user","timestamp":"2024-01-01T10:00:00Z","uuid":"u1","sessionId":"test-session-123","cwd":"/test/project","message":{"content":"Hello"}} +{"type":"assistant","timestamp":"2024-01-01T10:00:01Z","uuid":"a1","parentUuid":"u1","message":{"content":[{"type":"tool_use","id":"t0","name":"Write","input":{"file_path":"/test/project/main.go","content":"x"}}]}} +` + if err := os.WriteFile(filepath.Join(projectDir, "test-session-123.jsonl"), []byte(earlier), 0644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(projectDir, "later-456.jsonl"), []byte(later), 0644); err != nil { + t.Fatal(err) + } + setTestBackend(dir) + + req := httptest.NewRequest("GET", "/api/related/-test-project/later-456", nil) + w := httptest.NewRecorder() + handleAPIRelated(w, req) + if w.Code != http.StatusOK { + t.Fatalf("status %d: %s", w.Code, w.Body.String()) + } + var resp struct { + Kind string `json:"kind"` + Total int `json:"total"` + Related []struct { + SessionID string `json:"session_id"` + Strength string `json:"strength"` + Relations []struct { + Kind string `json:"kind"` + Paths []string `json:"paths"` + } `json:"relations"` + } `json:"related"` + } + if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil { + t.Fatalf("decode: %v: %s", err, w.Body.String()) + } + if resp.Kind != "ccx.related.v1" || resp.Total != 1 || len(resp.Related) != 1 { + t.Fatalf("envelope: %+v", resp) + } + r := resp.Related[0] + if r.SessionID != "test-session-123" || r.Strength != "medium" { + t.Fatalf("related: %+v", r) + } + kinds := map[string]bool{} + for _, rel := range r.Relations { + kinds[rel.Kind] = true + } + if !kinds["builds_on"] || !kinds["previous"] { + t.Fatalf("relations: %+v", r.Relations) + } + + // Unknown session -> 404; malformed path -> 404. + w = httptest.NewRecorder() + handleAPIRelated(w, httptest.NewRequest("GET", "/api/related/-test-project/nope", nil)) + if w.Code != http.StatusNotFound { + t.Fatalf("unknown session status %d", w.Code) + } + w = httptest.NewRecorder() + handleAPIRelated(w, httptest.NewRequest("GET", "/api/related/only-one-part", nil)) + if w.Code != http.StatusNotFound { + t.Fatalf("malformed path status %d", w.Code) + } +} diff --git a/skills/ccx-recap/SKILL.md b/skills/ccx-recap/SKILL.md index 181ae03..ba72262 100644 --- a/skills/ccx-recap/SKILL.md +++ b/skills/ccx-recap/SKILL.md @@ -40,10 +40,16 @@ its own CLI surface. fits; read it whole. Steps are the agent's own narration at the moment it acted — that sequence IS the story skeleton. 2. **Pick what matters.** Turns with edits, errors, linked commits, - high cost, or user pushback. Ignore command noise (`is_command`). + high cost, user pushback, or human interventions (`interrupts` — + the human pressed stop; `denials` — a tool call refused at the + permission prompt; both are on turns, steps, and stats). Ignore + command noise (`is_command`). 3. **Drill only there.** `ccx trace --turn N` for full narration, - mutations, and error attribution. Raw text when needed: the - `anchor_id` / `message_id` point into the session file. + mutations, and error attribution. Raw text when needed: `ccx view + --at ` opens the cited message in context. + When the story crosses sessions (a handoff picked up, a fork, a + parallel agent), `ccx related ` says which sessions and + how, with evidence — cite those anchors, do not guess the link. 4. **Verify before claiming.** "Done" requires evidence: a passing test step, a linked commit, a verification narration. Tag every important claim: diff --git a/skills/ccx-retro/SKILL.md b/skills/ccx-retro/SKILL.md index 55bd6cb..a8dcf18 100644 --- a/skills/ccx-retro/SKILL.md +++ b/skills/ccx-retro/SKILL.md @@ -32,12 +32,17 @@ matching its own CLI surface. - turns where the user pushed back — read the actual `user_text` and judge with your own comprehension; there is no correction flag, because keyword matching lies + - `interrupts` and `denials` (turn/step/stats): the human pressed + stop or refused a tool call — the sharpest correction signal in + the log; the denied call's evidence is marked `denied` - cost spikes without matching edits (spinning) - `warnings` (evidence gaps are findings too) -2. **Drill.** `ccx trace --turn N` on each suspect. Reconstruct: +2. **Drill.** `ccx trace --turn N` on each suspect; `ccx view + --at ` for the exact exchange. Reconstruct: what did the agent believe, what was actually true, what evidence was available at the time, what finally corrected it (user, test, - self-check)? + self-check)? If the mistake was inherited from an earlier session + (a stale handoff, a fork), `ccx related ` finds it. 3. **Classify each finding.** - `mistake` — agent had the evidence and got it wrong - `friction` — environment/tooling failed the agent diff --git a/skills/ccx/SKILL.md b/skills/ccx/SKILL.md index 7495ecb..ebcfbdd 100644 --- a/skills/ccx/SKILL.md +++ b/skills/ccx/SKILL.md @@ -25,6 +25,7 @@ ccx │ └── --goal SLUG # Filter by launch-receipt goal (deva --goal) ├── view [session] # View session in terminal │ └── --brief # Conversation only +│ └── --at MSG_ID [--context N] # Walk from a citation (search --hits / trace message_id) to its context ├── export [session] # Export session │ └── --format html|md|org|exec │ └── --shape full|brief|trace|exchange|human @@ -36,12 +37,16 @@ ccx ├── trace [session] # What the agent did: turn/step outline │ └── --json # Outline as JSON (ccx.outline.v1) │ └── --turn N # Full evidence for one turn (ccx.turn.v1) -│ └── --full # Complete trace bundle (ccx.trace.v2, large) +│ └── --full # Complete trace bundle (ccx.trace.v2, large; includes related) +├── related [session] # Which sessions connect to this one, and how +│ └── --json # Every relation with evidence (ccx.related.v1) ├── log [project] # Slice raw session logs by time scope │ └── --scope today|yesterday|week|month|quarter|year │ └── --since / --until TIME # RFC3339 or YYYY-MM-DD │ └── --tz ZONE # IANA timezone, UTC, local, or offset like +8 │ └── --json --raw # Evidence bundle, optional raw JSONL +│ └── --kind K1,K2 # Only these record kinds (user_prompt, assistant_message, tool_call, ...) +│ └── --match PHRASE [-w] # Only records whose raw line contains the phrase ├── insight [project] # HTML/JSON data report from session logs │ └── --scope --tz --since --until --all │ └── --json # Aggregates: days[]/providers[]/workspaces[] @@ -70,8 +75,12 @@ ccx view abc123 # View by session ID (prefix match) ccx export -f html # Export to HTML ccx sessions --scope yesterday --tz +8 --all --json # Session containers by end time ccx log --scope yesterday --tz +8 --all --json +ccx log --scope today --all --kind user_prompt # The humans in the loop: every prompt today, all sessions, in order +ccx log --scope today --all --kind interrupt,tool_denied # Where humans stopped or refused the agent +ccx log --scope month --all --match deadman -w # When a term came up inside a window (raw-line match) ccx trace # Outline of the latest workspace session ccx trace abc123 --turn 5 # Full evidence for one turn +ccx related abc123 # Sessions connected to this one: fork, handoff, mentions, shared files ccx web # Start web UI at localhost:8080 ``` @@ -79,13 +88,27 @@ ccx web # Start web UI at localhost:8080 `ccx trace` prints a terminal-readable outline: every turn (user intent) broken into steps (the agent's own narration), with tool, -edit, error, active-time, and cost rollups. Read the outline whole, +edit, error, interrupt/denial (human interventions), active-time, and +cost rollups. Read the outline whole, then drill: `--turn N` for one turn's full evidence, `--full` for the complete bundle. JSON kinds and field semantics are documented in `docs/schema.md`; header times are local (stated as `times UTC+X`), JSON times are UTC. Session IDs resolve across all projects automatically; `ccx trace ` works from any directory. +## Related: connections between sessions + +`ccx related [session]` joins the islands: which other sessions of the +workspace connect to this one and how — `forked_from`/`fork_of` +(shared message ids), `mentions`/`mentioned_by` (a session id named +in text), `handoff_from`/`handoff_to` (a baton file such as HANDOFF.md, +handoffs/, devlog written by one and read by the other later), +`builds_on`/`built_on_by` (a file edited by one, then touched by the +other), `overlaps` (concurrent), `previous`/`next`. Strength is a band +(strong/medium/weak). `--json` carries the evidence per relation +(session, message id, time, path, quote) — cite from that when a +claim spans sessions. Also present as `related` in `ccx trace --full`. + ## Multi-Provider ```bash @@ -152,8 +175,23 @@ Features: ccx search "error handling" # All providers ccx search --provider=cc "auth bug" # Claude Code only ccx search --after=2026-03-01 "deploy" # Date filtered +ccx search --content "deploy" # Also scan conversation text (user + assistant) +ccx search --content -w --sort first X # Whole word only, earliest mention first (FIRST column) +ccx search --content -w --hits X # Every mention as a citation: time, session, role, message id, quote +ccx search --raw "deploy" # Grep parity over raw transcript lines ``` +`-w` matters when the term prefixes a common word ("semantica" vs +"semantically"); `--json` carries `matches`, `previews`, `first_hit`. +`--content` also scans prompt history (`~/.claude/history.jsonl`, +`~/.codex/history.jsonl`): type `prompt` rows are prompts whose +session file is gone — the only evidence past session cleanup, so +"when did we first say X" reaches back further than the sessions do. +Cite from `--hits --json` (`message_id`, `time`, `quote`) rather than +from a session-level count when a claim needs evidence; walk to the +context with `ccx view --at ` (web: +`/session//#msg-`). + Web search supports provider prefixes: `cc: auth bug`, `cx: codex query`, `gx: grok query` ## Fork Session