feat: session connections, citations, and the human in the loop

CHANGELOG.md

@@ -7,6 +7,31 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
 
 ## [Unreleased]
 
+### Added
+- **`ccx related [session]`: which sessions connect to this one, and how.** Sessions were islands; the joins were in the transcripts but nothing computed them. `related` derives, deterministically and with evidence, the anchor session's connections to the other sessions of its workspace: `forked_from`/`fork_of` (the transcripts share message ids — Claude Code fork and `ccx fork` copy history verbatim), `mentions`/`mentioned_by` (a session id named in conversation text, quoted), `handoff_from`/`handoff_to` (a baton file — HANDOFF.md, handoffs/, devlog, PLAN.md — written by one and read by the other later), `builds_on`/`built_on_by` (a workspace file edited by one, then read or edited by the other), `overlaps` (concurrent), `previous`/`next`. Strength is a band (strong/medium/weak), never a score; path lists are capped with the count kept and `truncated` set; `--json` (`ccx.related.v1`) carries message id, time, path, and quote per relation. `ccx trace --full` gains the same list as `related`. Design: docs/design/0006-session-connections.md.
+- **`ccx search -w/--word` matches whole words.** Matching was substring-only, so a term that prefixes a common word was unanswerable: `search --content semantica` returned 47 sessions, 46 of them "semantic*ally*", and ccx alone could not tell 0 real hits from 46 (docs/devlog/2026-08-18-search-word-boundary-dogfood.org). `-w` demands an ASCII word boundary on each side of the query that starts/ends with a word character (so "semantica-agi" and "(semantica)" still hit; CJK queries are unaffected) and applies to names, summaries, conversation text, and `--raw` lines alike.
+- **`ccx search --hits` turns matches into citations.** One row per matching message — time, session, role, message id, quote — oldest first across sessions, `-n`-capped with a visible "showing N of M". The anchors are the same ones `trace` and `view` use, so a claim built on a search can point at its evidence (design: docs/design/0005-evidence-citations-lessons-from-semantica.md). Under `--raw` the unit is a transcript line, anchored by its own `uuid`/`type`/`timestamp`.
+- **`ccx search --content` reports when: `FIRST` column, `first_hit` in `--json`, `--sort first|last|hits`.** "When did we first mention X" needs the earliest matching message and an oldest-first order; results only carried session end time and ranked by hit count. Each content hit now records the timestamp of its earliest matching message (parsed messages by default; the raw line's top-level `timestamp` under `--raw`), printed as `FIRST` and sortable with `--sort first`; `--sort last` orders by session activity; `--sort hits` is the old order and the default.
+- **Codex interruptions count too.** `event_msg.turn_aborted` with reason `interrupted` (144 in one store) is Codex's "[Request interrupted by user]"; it now parses as the same `interrupt` kind, so `trace` interrupt counts and `log --kind interrupt` cover both providers.
+- **Human interventions are first-class in `trace` and `log`.** "[Request interrupted by user]" (the human pressed stop) and permission-prompt rejections ("The user doesn't want to proceed with this tool use") are the human in the loop, but ccx read the first as a *prompt* — it opened a fake turn `u: [Request interrupted by user for tool use]` — and the second as an ordinary tool error. New parser kind `interrupt` (harness marker, never an exchange anchor) and `parser.IsToolDenial`; `trace` counts `interrupts` and `denials` on turn, step, and stats, marks the rejected call `denied` (not an error, it did not run), and badges them in the outline header and rows (`1 interrupt, 2 denied`; step `[4t 1! 1d]`); `log` reports kinds `interrupt` and `tool_denied` (`ccx log --scope today --all --kind interrupt,tool_denied`). 650 interruptions across 321 sessions and 145 rejections in one real store were invisible before.
+- **`GET /api/related/<project>/<session>` serves session connections as JSON** — the `ccx.related.v1` envelope (`related`, `total`, `shown`, `warnings`; `?limit=N`), computed by the same `trace.RelateWorkspace` the CLI uses, so a web panel or an agent reading the API sees exactly what `ccx related --json` prints. Fetched on demand: it costs a parse of every workspace session (cached after the first call).
+- **`ccx search --content` scans prompt history.** Claude Code and Codex append every human prompt to `history.jsonl`, and those files outlive session cleanup (one real store: prompts back to 2025-09-28, sessions back to 2025-12-07). Matches surface as type `prompt` — only for prompts whose session file is no longer in the store, so a prompt never appears twice — with `FIRST`, a `[user]` quote, and under `--hits` a `history:line` anchor. "When did we first say X" now reaches past the session horizon.
+- **`ccx view --at MESSAGE_ID [--context N]` walks from a citation to its context.** Search `--hits` and `trace` hand out message ids, but nothing in the CLI could open one; drill-down meant the web page or raw grep (open since docs/devlog/2026-08-03-content-search-noise.org finding 4). `--at` renders the cited message with N messages before and after it (wire order, flattened; the target survives `--brief`), and says where it sits: `message 1 of 763`. Prefixes resolve; ambiguous prefixes are an error, not a guess.
+
+### Changed
+- **`ccx trace` labels narration-less steps.** A step the agent never narrated (straight to tools) rendered as a bare badge row `1.   [4t]`; the outline now says what it ran: `(no narration) Bash x3, Read` (docs/devlog/2026-08-17-codex-0147-rollout-drift.org finding 3).
+- **`ccx log --kind` and `--match PHRASE [-w]` turn the firehose into a timeline.** `log` emitted every record in scope (16k for one day) with no way to narrow; "what did the humans ask today, across every session" was not answerable. `--kind user_prompt,assistant_message` keeps only those kinds; `--match` keeps records whose raw transcript line contains the phrase (grep parity, `-w` for whole words) — the time-bounded complement of `search --hits`. Metrics stay honest: `records` is scope-wide, new `records_matched` is the narrowed count, `showing` is after `-n`.
+- **`ccx search --content` is ~10x faster and shows progress.** The scan ran on one core and lowercased every transcript line: 6m18s cold / 1m14s warm over a 3.5 GB store, silent throughout. Sessions now scan on a bounded worker pool (up to 8), the raw prefilter matches case-insensitively without allocating and stops at the first hit, and a `scanning N/M sessions` line ticks on stderr when it is a terminal. Same store, warm: 7.6s.
+
+### Fixed
+- **`ccx log` now applies the same conversation rules as the parser.** Two classes of user-role lines were reported as `user_prompt`: Claude harness wrappers (slash-command markers, `<local-command-*>` echoes, task notifications), injected meta messages (skill bodies), and compaction carriers — 233 "prompts" today of which 79 were typed by a human; and Codex 0.147 raw `response_item` messages, including the injected AGENTS.md envelope, while the real `item_completed` UserMessage/AgentMessage rows rendered as bare `item_completed` with no text. `sessionlog` now reuses `parser.ClassifyUserText` and the Codex TurnItem decoder (`DecodeCompletedTurnMessage`, exported), demotes legacy/raw duplicates in a 0.147 rollout to `legacy_message`/`model_input`/`model_output` (records kept, counts fixed), and re-tallies session kinds and preview after demotion. `user_prompts`/`assistant_messages` metrics and `insight` reports built on them count the conversation once.
+- **`ccx log` previews Claude tool results.** A `tool_result` block's payload lives under `content`; the preview only looked at `text`/`message`, so every Claude tool result row read as the literal word `tool_result`.
+- **Heredoc bodies no longer count as shell redirects.** `extractRedirectPaths` scanned the whole Bash command, so a Go `if n > 0` or a markdown `> 2026-08-18` inside `python3 - <<'EOF'` / `cat > f <<'EOF'` became "edited files" (`.../0`, `.../2026-08-18`) in `trace` `files_edited` and in session connections. Heredoc bodies are stripped before the redirect scan.
+- **`-n` is the `--limit` shorthand everywhere.** Only `search` had it; `sessions -n 2` failed with "unknown shorthand flag". `sessions`, `projects`, and `log` now accept `-n` too.
+- **A session whose summary matched dropped its content evidence.** The summary hit short-circuited the content scan, so under `--content` the session where the term was actually discussed could be the one result with no hit count, previews, or first-hit time. Summary hits stay typed `session` but now carry the content fields.
+- **Codex 0.147 conversations render again.** Codex moved its UI-facing user and assistant records from legacy `event_msg.user_message` / `agent_message` events to canonical `event_msg.item_completed` TurnItems. The Codex adapter now selects one conversation source per rollout, reads stable `UserMessage` / `AgentMessage` item IDs and content, keeps legacy rollouts working, and never mistakes raw `response_item` model input (which can contain injected instruction envelopes) for a human prompt. Discovery metadata, terminal view, web, export, search, and trace now agree; the parse-cache format is bumped so upgrades cannot serve blank cached sessions.
+- **`search --hits` citations print the whole message id.** The MESSAGE column cut every id to 8 characters. That is unambiguous for a uuid and wrong for the synthetic ids: `codex-thinking-90` and `codex-thinking-331` both printed as `codex-th`, and a prompt-history anchor at line 4426 printed as `line:442` — a line that exists, and is not the one cited. The citation was not merely unpasteable into `ccx view --at`; it pointed elsewhere. Only hex-prefixed ids are shortened now.
+
 ## [0.15.0] - 2026-08-11
 
 ### Added

README.md

@@ -75,11 +75,15 @@ ccx sessions --provider=cx       # Codex sessions in this workspace
 ccx sessions --after=2026-03-01  # Date filtered
 ccx sessions --scope yesterday --tz +8 --all --json # Session containers by end time
 ccx view [session]               # View in terminal
+ccx view [session] --at MSG_ID   # Around one cited message (search --hits / trace message_id)
 ccx export --shape brief         # Export conversation-only HTML
 ccx export --shape human         # Only the human's turns, citable
 ccx trace [session] -o trace.json # Extract evidence for context folding
+ccx related [session]            # Which sessions connect to this one (fork, handoff, mentions, shared files)
 ccx log --scope yesterday --tz +8 --all --json # Time-sliced log evidence
 ccx search "auth bug"            # Search across sessions + memory
+ccx search --content -w --sort first goose # Whole-word content hits, earliest first
+ccx search --content -w --hits goose      # Every mention, quoted + anchored (time, session, message id)
 ccx run ccx-recap --agent claude # Run a bundled skill via an agent CLI
 ccx fork abc123                  # Fork session to current project
 ccx doctor                       # Check setup

docs/design/0004-transcript-adapter-contract.md

@@ -0,0 +1,115 @@
+# Transcript adapter contract
+
+ccx currently supports Claude Code, Codex, and Grok. The next target set
+also includes cctrace, Gemini CLI, Kimi Code, opencode, pi, dsh, and
+Cursor. That is not a request for seven more ad hoc parsers. Some sources
+are conversations, some may be lineage/orchestration records, and some
+may be derived evidence. Each source needs a discovery spike before ccx
+classifies it.
+
+The Codex 0.147 break exposed the architectural rule: provider wire
+formats churn; renderers must depend on a stable ccx transcript contract,
+and format drift must be visible rather than becoming an empty page.
+
+## Layers
+
+```text
+provider homes / indexes / files
+        |
+        v
+provider-native reader + versioned wire adapter
+        |
+        v
+ccx Transcript + Lineage + Diagnostics
+        |
+        +--> legacy Message-tree projection (migration only)
+        +--> turn/step analysis
+        +--> terminal, web, export, search
+```
+
+Provider-native structs stay inside `internal/provider/<id>`. Do not add
+one cross-provider union of every upstream event type. Shared code begins
+after native records have been reduced to ccx semantics.
+
+## Stable domain
+
+The provider-neutral transcript needs these concepts explicitly:
+
+- `SessionRef`: identity, provider, source path, time bounds, cwd, model,
+  source format/version, archive state.
+- `Transcript`: ordered turns/blocks plus a declared topology.
+- `Topology`: linear turns, message tree, or linked child sessions. Do
+  not force Claude branches and Codex child threads into one tree.
+- `Block`: human text, assistant text, reasoning summary, tool call,
+  tool result, compaction, system/meta, attachment, unknown.
+- `Lineage`: parent/child session edges separate from transcript nesting.
+- `Capabilities`: whether tokens, cost, errors, files, reasoning,
+  branching, and resume metadata are actually reported. Missing is not
+  zero.
+- `Diagnostics`: malformed records, unknown variants, unsupported format,
+  dropped records, and lossy projections. Diagnostics are findings and
+  must reach CLI/web consumers.
+
+`parser.Session` / `parser.Message` is currently the renderer contract and
+is still Claude-shaped. Keep it as a compatibility projection while the
+neutral domain lands; do not make it the native model for the next
+provider.
+
+## Adapter rules
+
+1. Select an authoritative upstream source for each semantic class.
+   Model request/response history, UI events, hooks, and tool lifecycle
+   records are not interchangeable.
+2. Record the observed source format and producer version. Prefer a real
+   format discriminant; otherwise use explicit feature detection.
+3. Unknown variants survive as diagnostics or `unknown` blocks. Never
+   silently turn a non-empty source into an empty transcript.
+4. Deduplicate by upstream stable ID. Timestamp/text heuristics are a
+   documented legacy fallback only.
+5. Preserve raw source anchors without exposing raw private envelopes in
+   rendered conversation text.
+6. Keep discovery quick parse and full parse semantically identical.
+7. Do not infer cost, error state, or token semantics when the provider
+   does not report them.
+8. Provider homes remain read-only.
+
+## Provider acceptance gate
+
+Support is not complete until one sanitized fixture set and one shared
+contract suite prove all applicable surfaces:
+
+- format reference names producer version and observed artifacts;
+- fixtures cover plain chat, tools, compaction/resume, and lineage where
+  the provider has them;
+- list metadata equals full-parse metadata;
+- terminal view, web, export, search, and trace use the same transcript;
+- unknown format/variant tests produce visible diagnostics;
+- raw instruction, auth, and environment envelopes do not render as
+  human turns;
+- no unsupported metric is displayed as measured zero;
+- a read-only live smoke passes before release.
+
+The schema audit command must dispatch through the same provider adapter
+registry. A separate hand-maintained Claude field list cannot enforce this
+gate.
+
+## Sequence
+
+1. Land the Codex 0.147 adapter fix and regression fixture.
+2. Introduce `Transcript`, `Topology`, `Capabilities`, and `Diagnostics`,
+   plus a projection back to the current message tree.
+3. Move Claude Code, Codex, and Grok behind the contract without changing
+   their rendered output; add the shared acceptance suite.
+4. Replace the Claude-only schema audit with provider-dispatched audits.
+5. Spike each remaining target against real local artifacts, write its
+   format reference, sanitize fixtures, then implement one adapter at a
+   time. Do not promise verbs before the fixture proves the data exists.
+6. Model dsh/cctrace orchestration or derived evidence as lineage/source
+   layers if their artifacts are not native conversations; do not fake
+   them into chat messages.
+
+Target CLI versions observed for this planning snapshot (2026-08-17) are
+inputs to the spikes, not compatibility promises: Claude Code 2.1.234,
+cctrace 0.40.0, Codex 0.147.0, Gemini CLI 0.55.1, Grok CLI 1.0.5,
+Kimi Code 0.36.1, opencode 1.18.18, pi 0.84.2, dsh 0.1.0-rc.7, and
+Cursor 2026.08.11-e8db854.