From d1e009ad8f9a636a71586947ec9071a3bba9f05f Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 03:16:42 -0700 Subject: [PATCH 01/13] fix(codex): render 0.147 conversations via completed TurnItems Codex 0.147 moved its UI-facing user and assistant records from the legacy event_msg.user_message / agent_message events to canonical event_msg.item_completed TurnItems. ccx listed those sessions with (no summary), zero messages, and a blank web conversation while tool calls still rendered, which read as a renderer bug and was a source selection regression. - Add a Codex-native TurnItem wire adapter (turn_items.go). - Full parse detects completed conversation items first, then picks one message source for the whole rollout so hybrid files cannot render both. Quick parse counts both variants in a single scan and prefers the completed totals and summary. - Keep ignoring raw response_item.message: its user-role records can carry injected instruction and environment envelopes that would show up as human prompts. - Use completed item IDs as message UUIDs so export and drill-down anchors stay stable. - Bump CacheFormatVersion 3 -> 4 so an upgraded binary cannot keep serving blank cached parses. Fixture is synthetic and sanitized; covers canonical selection, legacy/hybrid dedup, count and summary parity, stable IDs, model metadata, and instruction-envelope exclusion. Co-Authored-By: Claude Opus 5 --- .../0004-transcript-adapter-contract.md | 115 ++++++++++++++++++ .../2026-08-17-codex-0147-rollout-drift.org | 68 +++++++++++ internal/parser/types.go | 2 +- internal/provider/codex/backend.go | 92 +++++++++++++- internal/provider/codex/backend_0147_test.go | 87 +++++++++++++ internal/provider/codex/turn_items.go | 105 ++++++++++++++++ 6 files changed, 465 insertions(+), 4 deletions(-) create mode 100644 docs/design/0004-transcript-adapter-contract.md create mode 100644 docs/devlog/2026-08-17-codex-0147-rollout-drift.org create mode 100644 internal/provider/codex/backend_0147_test.go create mode 100644 internal/provider/codex/turn_items.go diff --git a/docs/design/0004-transcript-adapter-contract.md b/docs/design/0004-transcript-adapter-contract.md new file mode 100644 index 0000000..1843b4b --- /dev/null +++ b/docs/design/0004-transcript-adapter-contract.md @@ -0,0 +1,115 @@ +# Transcript adapter contract + +ccx currently supports Claude Code, Codex, and Grok. The next target set +also includes cctrace, Gemini CLI, Kimi Code, opencode, pi, dsh, and +Cursor. That is not a request for seven more ad hoc parsers. Some sources +are conversations, some may be lineage/orchestration records, and some +may be derived evidence. Each source needs a discovery spike before ccx +classifies it. + +The Codex 0.147 break exposed the architectural rule: provider wire +formats churn; renderers must depend on a stable ccx transcript contract, +and format drift must be visible rather than becoming an empty page. + +## Layers + +```text +provider homes / indexes / files + | + v +provider-native reader + versioned wire adapter + | + v +ccx Transcript + Lineage + Diagnostics + | + +--> legacy Message-tree projection (migration only) + +--> turn/step analysis + +--> terminal, web, export, search +``` + +Provider-native structs stay inside `internal/provider/`. Do not add +one cross-provider union of every upstream event type. Shared code begins +after native records have been reduced to ccx semantics. + +## Stable domain + +The provider-neutral transcript needs these concepts explicitly: + +- `SessionRef`: identity, provider, source path, time bounds, cwd, model, + source format/version, archive state. +- `Transcript`: ordered turns/blocks plus a declared topology. +- `Topology`: linear turns, message tree, or linked child sessions. Do + not force Claude branches and Codex child threads into one tree. +- `Block`: human text, assistant text, reasoning summary, tool call, + tool result, compaction, system/meta, attachment, unknown. +- `Lineage`: parent/child session edges separate from transcript nesting. +- `Capabilities`: whether tokens, cost, errors, files, reasoning, + branching, and resume metadata are actually reported. Missing is not + zero. +- `Diagnostics`: malformed records, unknown variants, unsupported format, + dropped records, and lossy projections. Diagnostics are findings and + must reach CLI/web consumers. + +`parser.Session` / `parser.Message` is currently the renderer contract and +is still Claude-shaped. Keep it as a compatibility projection while the +neutral domain lands; do not make it the native model for the next +provider. + +## Adapter rules + +1. Select an authoritative upstream source for each semantic class. + Model request/response history, UI events, hooks, and tool lifecycle + records are not interchangeable. +2. Record the observed source format and producer version. Prefer a real + format discriminant; otherwise use explicit feature detection. +3. Unknown variants survive as diagnostics or `unknown` blocks. Never + silently turn a non-empty source into an empty transcript. +4. Deduplicate by upstream stable ID. Timestamp/text heuristics are a + documented legacy fallback only. +5. Preserve raw source anchors without exposing raw private envelopes in + rendered conversation text. +6. Keep discovery quick parse and full parse semantically identical. +7. Do not infer cost, error state, or token semantics when the provider + does not report them. +8. Provider homes remain read-only. + +## Provider acceptance gate + +Support is not complete until one sanitized fixture set and one shared +contract suite prove all applicable surfaces: + +- format reference names producer version and observed artifacts; +- fixtures cover plain chat, tools, compaction/resume, and lineage where + the provider has them; +- list metadata equals full-parse metadata; +- terminal view, web, export, search, and trace use the same transcript; +- unknown format/variant tests produce visible diagnostics; +- raw instruction, auth, and environment envelopes do not render as + human turns; +- no unsupported metric is displayed as measured zero; +- a read-only live smoke passes before release. + +The schema audit command must dispatch through the same provider adapter +registry. A separate hand-maintained Claude field list cannot enforce this +gate. + +## Sequence + +1. Land the Codex 0.147 adapter fix and regression fixture. +2. Introduce `Transcript`, `Topology`, `Capabilities`, and `Diagnostics`, + plus a projection back to the current message tree. +3. Move Claude Code, Codex, and Grok behind the contract without changing + their rendered output; add the shared acceptance suite. +4. Replace the Claude-only schema audit with provider-dispatched audits. +5. Spike each remaining target against real local artifacts, write its + format reference, sanitize fixtures, then implement one adapter at a + time. Do not promise verbs before the fixture proves the data exists. +6. Model dsh/cctrace orchestration or derived evidence as lineage/source + layers if their artifacts are not native conversations; do not fake + them into chat messages. + +Target CLI versions observed for this planning snapshot (2026-08-17) are +inputs to the spikes, not compatibility promises: Claude Code 2.1.234, +cctrace 0.40.0, Codex 0.147.0, Gemini CLI 0.55.1, Grok CLI 1.0.5, +Kimi Code 0.36.1, opencode 1.18.18, pi 0.84.2, dsh 0.1.0-rc.7, and +Cursor 2026.08.11-e8db854. diff --git a/docs/devlog/2026-08-17-codex-0147-rollout-drift.org b/docs/devlog/2026-08-17-codex-0147-rollout-drift.org new file mode 100644 index 0000000..f3190a1 --- /dev/null +++ b/docs/devlog/2026-08-17-codex-0147-rollout-drift.org @@ -0,0 +1,68 @@ +* [2026-08-17] Dev Log: Codex 0.147 rollout drift :CODEX:RELIABILITY: + +** Context +Codex 0.147 sessions appeared in ccx's lists but had =(no summary)=, +zero messages, zero turns, and a blank web conversation. Tool calls +still appeared, which made the failure look like a renderer bug. + +** Evidence +- The rollout still uses =session_meta=, =turn_context=, =event_msg=, + and =response_item= top-level records. +- The legacy =event_msg.user_message= and =agent_message= variants are + absent from a native 0.147 rollout. +- Visible conversation records are now =event_msg.item_completed= + carrying =TurnItem::UserMessage= and =TurnItem::AgentMessage=. +- Those items have stable IDs and typed content arrays. User text uses + =type: text=; agent text currently uses =type: Text=. +- =response_item.message= is not the visible transcript contract. Its + user-role records can include injected instruction and environment + envelopes. Rendering those would expose harness context as if the + human typed it. + +This was a parser/source-selection regression, not an HTML problem. + +** Fix +- Added a Codex-native TurnItem wire adapter in + =internal/provider/codex/turn_items.go=. +- Full parse detects whether a rollout contains completed conversation + TurnItems, then selects exactly one message source for the whole + parse: completed TurnItems when present, legacy message events + otherwise. This prevents hybrid/migrated rollouts from rendering both. +- Quick parse counts legacy and completed messages separately in one + scan, then selects the completed totals and summary when present. +- Raw =response_item.message= records remain deliberately ignored; + response-item tool calls remain supported. +- Completed item IDs become ccx message UUIDs instead of synthetic line + IDs, preserving stable anchors for export and drill-down. +- Parse cache format bumped from 3 to 4. + +** Tests +- Sanitized 0.147 fixture-in-code covers canonical item selection, + legacy/hybrid deduplication, summary and count parity, stable IDs, + model metadata, and instruction-envelope exclusion. +- Existing Codex provider tests cover the legacy fallback. +- Live read-only smoke covered =sessions=, =view --brief=, =trace=, + markdown export, the global sessions API, and the web session route. + +** Dogfood findings +1. *Schema drift fails silently.* An unsupported conversation variant + degrades to =(no summary)= and zero messages with no parser warning. + This fix restores 0.147 but does not make the next drift loud. + Fix direction: provider parse diagnostics on =Session=, surfaced in + CLI/web, with unknown top-level/payload/item variant counts. +2. *=ccx-audit-schema= is Claude-only.* The command audits Claude's + =rawMessage= fields and cannot audit Codex or Grok despite the + multi-provider product contract. Fix direction: a provider-dispatched + audit interface sharing each adapter's known wire variants. +3. *=trace= can emit an unlabeled tool-only step.* The outline showed a + numbered step containing only a tool-count badge. Fix direction: + derive a bounded label from the tool sequence or attach it to the + adjacent narration step. + +The first two findings belong to the transcript-adapter work, not as +more conditionals in the Codex parser. See +=docs/design/0004-transcript-adapter-contract.md=. + +** Privacy +No real session ID, transcript text, home path, raw rollout, or cost is +committed. The regression fixture is synthetic and sanitized. diff --git a/internal/parser/types.go b/internal/parser/types.go index e0f9d33..0e57882 100644 --- a/internal/parser/types.go +++ b/internal/parser/types.go @@ -37,7 +37,7 @@ type Project struct { // struct versions silently, so without this stamp an upgraded binary // keeps serving parses produced by the old code until the source // session file itself happens to change. -const CacheFormatVersion = 3 +const CacheFormatVersion = 4 type Session struct { ID string diff --git a/internal/provider/codex/backend.go b/internal/provider/codex/backend.go index 8aa3519..2805130 100644 --- a/internal/provider/codex/backend.go +++ b/internal/provider/codex/backend.go @@ -545,6 +545,12 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { var firstTime time.Time var lastTime time.Time seenToolCallIDs := make(map[string]bool) + seenCompletedMessageIDs := make(map[string]bool) + legacyMessageCount := 0 + legacyUserPrompts := 0 + completedMessageCount := 0 + completedUserPrompts := 0 + completedSummary := "" countToolCall := func(callID string) { callID = strings.TrimSpace(callID) @@ -610,8 +616,8 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { switch header.Type { case "user_message": - stats.MessageCount++ - stats.UserPrompts++ + legacyMessageCount++ + legacyUserPrompts++ var payload userMessagePayload if err := json.Unmarshal(rollout.Payload, &payload); err == nil && summary == "" { @@ -619,7 +625,23 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { } case "agent_message": - stats.MessageCount++ + legacyMessageCount++ + + case "item_completed": + message, ok := decodeCompletedTurnMessage(rollout.Payload) + if !ok || (message.ID != "" && seenCompletedMessageIDs[message.ID]) { + continue + } + if message.ID != "" { + seenCompletedMessageIDs[message.ID] = true + } + completedMessageCount++ + if message.Role == "user" { + completedUserPrompts++ + if completedSummary == "" { + completedSummary = stripUserMessagePrefix(message.Text) + } + } case "exec_command_end": var payload execCommandEndPayload @@ -727,6 +749,14 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { if err := scanner.Err(); err != nil { return nil, err } + if completedMessageCount > 0 { + stats.MessageCount = completedMessageCount + stats.UserPrompts = completedUserPrompts + summary = completedSummary + } else { + stats.MessageCount = legacyMessageCount + stats.UserPrompts = legacyUserPrompts + } if firstTime.IsZero() || lastTime.IsZero() { if info, err := os.Stat(filePath); err == nil { @@ -757,6 +787,10 @@ func (b *Backend) quickParseSession(filePath string) (*parser.Session, error) { } func (b *Backend) parseSession(filePath string, threadNames map[string]string) (*parser.Session, error) { + useCompletedTurnMessages, err := hasCompletedTurnMessages(filePath) + if err != nil { + return nil, err + } file, err := os.Open(filePath) if err != nil { return nil, err @@ -774,6 +808,7 @@ func (b *Backend) parseSession(filePath string, threadNames map[string]string) ( var firstTime time.Time var lastTime time.Time var messages []*parser.Message + seenCompletedMessageIDs := make(map[string]bool) pendingTools := make(map[string]pendingToolCall) handledCallIDs := make(map[string]bool) completedCallIDs := make(map[string]bool) @@ -869,6 +904,9 @@ func (b *Backend) parseSession(filePath string, threadNames map[string]string) ( switch header.Type { case "user_message": + if useCompletedTurnMessages { + continue + } var payload userMessagePayload if err := json.Unmarshal(rollout.Payload, &payload); err != nil { continue @@ -904,6 +942,9 @@ func (b *Backend) parseSession(filePath string, threadNames map[string]string) ( )) case "agent_message": + if useCompletedTurnMessages { + continue + } var payload agentMessagePayload if err := json.Unmarshal(rollout.Payload, &payload); err != nil { continue @@ -918,6 +959,51 @@ func (b *Backend) parseSession(filePath string, threadNames map[string]string) ( parser.ContentBlock{Type: "text", Text: payload.Message}, )) + case "item_completed": + if !useCompletedTurnMessages { + continue + } + message, ok := decodeCompletedTurnMessage(rollout.Payload) + if !ok || (message.ID != "" && seenCompletedMessageIDs[message.ID]) { + continue + } + if message.ID != "" { + seenCompletedMessageIDs[message.ID] = true + } + messageID := message.ID + if messageID == "" { + messageID = fmt.Sprintf("codex-%s-%d", message.Role, lineNum) + } + if message.Role == "user" { + text := stripUserMessagePrefix(message.Text) + if text == "" { + text = "(empty)" + } + if firstUserSummary == "" { + firstUserSummary = text + } + stats.MessageCount++ + stats.UserPrompts++ + messages = append(messages, newMessage( + messageID, + "user", + parser.KindUserPrompt, + ts, + currentModel, + parser.ContentBlock{Type: "text", Text: text}, + )) + } else { + stats.MessageCount++ + messages = append(messages, newMessage( + messageID, + "assistant", + parser.KindAssistant, + ts, + currentModel, + parser.ContentBlock{Type: "text", Text: message.Text}, + )) + } + case "agent_reasoning", "agent_reasoning_raw_content": var payload agentReasoningPayload if err := json.Unmarshal(rollout.Payload, &payload); err != nil { diff --git a/internal/provider/codex/backend_0147_test.go b/internal/provider/codex/backend_0147_test.go new file mode 100644 index 0000000..bcb9a71 --- /dev/null +++ b/internal/provider/codex/backend_0147_test.go @@ -0,0 +1,87 @@ +package codex + +import ( + "path/filepath" + "testing" +) + +func TestQuickParseSessionCodex0147ItemCompletedMessages(t *testing.T) { + home := t.TempDir() + sessionsDir := filepath.Join(home, "sessions") + rolloutPath := filepath.Join(sessionsDir, "2026", "08", "17", "rollout-0147.jsonl") + writeRollout(t, rolloutPath, codex0147Rollout) + + backend := NewWithDirs(home, sessionsDir, filepath.Join(home, "archived_sessions")) + projects, err := backend.DiscoverProjects() + if err != nil { + t.Fatalf("DiscoverProjects() error = %v", err) + } + if len(projects) != 1 || len(projects[0].Sessions) != 1 { + t.Fatalf("discovered projects/sessions = %d/%d, want 1/1", len(projects), len(projects[0].Sessions)) + } + + session := projects[0].Sessions[0] + if session.Summary != "actual user prompt" { + t.Fatalf("Summary = %q, want actual user prompt", session.Summary) + } + if session.Stats.MessageCount != 2 { + t.Fatalf("MessageCount = %d, want 2", session.Stats.MessageCount) + } + if session.Stats.UserPrompts != 1 { + t.Fatalf("UserPrompts = %d, want 1", session.Stats.UserPrompts) + } + if session.Model != "gpt-5.6-sol" { + t.Fatalf("Model = %q, want gpt-5.6-sol", session.Model) + } +} + +func TestParseSessionCodex0147UsesCanonicalTurnItems(t *testing.T) { + home := t.TempDir() + sessionsDir := filepath.Join(home, "sessions") + rolloutPath := filepath.Join(sessionsDir, "2026", "08", "17", "rollout-0147.jsonl") + writeRollout(t, rolloutPath, codex0147Rollout) + + backend := NewWithDirs(home, sessionsDir, filepath.Join(home, "archived_sessions")) + session, err := backend.ParseSession(rolloutPath) + if err != nil { + t.Fatalf("ParseSession() error = %v", err) + } + + if session.Stats.MessageCount != 2 || session.Stats.UserPrompts != 1 { + t.Fatalf("message stats = %d/%d, want 2 messages and 1 prompt", + session.Stats.MessageCount, session.Stats.UserPrompts) + } + if len(session.RootMessages) != 1 { + t.Fatalf("RootMessages = %d, want 1", len(session.RootMessages)) + } + root := session.RootMessages[0] + if root.UUID != "user-item-1" { + t.Fatalf("user UUID = %q, want stable item id", root.UUID) + } + if got := root.Content[0].Text; got != "actual user prompt" { + t.Fatalf("user text = %q, want actual user prompt", got) + } + if len(root.Children) != 1 { + t.Fatalf("user children = %d, want 1", len(root.Children)) + } + assistant := root.Children[0] + if assistant.UUID != "assistant-item-1" { + t.Fatalf("assistant UUID = %q, want stable item id", assistant.UUID) + } + if got := assistant.Content[0].Text; got != "actual assistant reply" { + t.Fatalf("assistant text = %q, want actual assistant reply", got) + } +} + +// Codex 0.147 persists provider request/response messages and canonical +// item_completed turn items together. The response_item user record includes +// injected instruction envelopes, so it must never become the visible prompt. +const codex0147Rollout = `{"timestamp":"2026-08-18T04:45:37.600Z","type":"session_meta","payload":{"id":"thread-0147","timestamp":"2026-08-18T04:45:37.600Z","cwd":"/tmp/work/project-0147","originator":"codex-tui","cli_version":"0.147.0","source":"cli","model_provider":"openai"},"ordinal":0} +{"timestamp":"2026-08-18T04:45:37.610Z","type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"private injected instruction envelope"}]},"ordinal":1} +{"timestamp":"2026-08-18T04:45:37.620Z","type":"turn_context","payload":{"turn_id":"turn-1","cwd":"/tmp/work/project-0147","model":"gpt-5.6-sol"},"ordinal":2} +{"timestamp":"2026-08-18T04:45:37.630Z","type":"event_msg","payload":{"type":"user_message","message":"legacy duplicate prompt","images":[],"local_images":[]},"ordinal":3} +{"timestamp":"2026-08-18T04:45:37.640Z","type":"event_msg","payload":{"type":"item_completed","thread_id":"thread-0147","turn_id":"turn-1","item":{"type":"UserMessage","id":"user-item-1","content":[{"type":"text","text":"actual user prompt","text_elements":[]}]},"completed_at_ms":1787028337640},"ordinal":4} +{"timestamp":"2026-08-18T04:45:37.650Z","type":"event_msg","payload":{"type":"agent_message","message":"legacy duplicate reply"},"ordinal":5} +{"timestamp":"2026-08-18T04:45:37.660Z","type":"response_item","payload":{"type":"message","id":"response-assistant-1","role":"assistant","phase":"final_answer","content":[{"type":"output_text","text":"actual assistant reply"}]},"ordinal":6} +{"timestamp":"2026-08-18T04:45:37.670Z","type":"event_msg","payload":{"type":"item_completed","thread_id":"thread-0147","turn_id":"turn-1","item":{"type":"AgentMessage","id":"assistant-item-1","content":[{"type":"Text","text":"actual assistant reply"}],"phase":"final_answer"},"completed_at_ms":1787028337670},"ordinal":7} +` diff --git a/internal/provider/codex/turn_items.go b/internal/provider/codex/turn_items.go new file mode 100644 index 0000000..155849e --- /dev/null +++ b/internal/provider/codex/turn_items.go @@ -0,0 +1,105 @@ +package codex + +import ( + "bufio" + "encoding/json" + "os" + "strings" +) + +// Codex 0.147 made item_completed TurnItems the persisted, UI-facing +// transcript. Raw response_item messages are model I/O and can contain injected +// developer/user envelopes, so they are not a safe conversation source. +type completedTurnMessage struct { + ID string + Role string + Text string +} + +type itemCompletedPayload struct { + Type string `json:"type"` + Item json.RawMessage `json:"item"` +} + +type turnItemMessagePayload struct { + ID string `json:"id"` + Type string `json:"type"` + Content []turnItemContent `json:"content"` +} + +type turnItemContent struct { + Type string `json:"type"` + Text string `json:"text"` + Name string `json:"name"` +} + +func hasCompletedTurnMessages(filePath string) (bool, error) { + file, err := os.Open(filePath) + if err != nil { + return false, err + } + defer file.Close() + + scanner := bufio.NewScanner(file) + scanner.Buffer(make([]byte, 0, 64*1024), maxScannerBufferBytes) + for scanner.Scan() { + var rollout rolloutLine + if err := json.Unmarshal(scanner.Bytes(), &rollout); err != nil || rollout.Type != "event_msg" { + continue + } + if _, ok := decodeCompletedTurnMessage(rollout.Payload); ok { + return true, nil + } + } + if err := scanner.Err(); err != nil { + return false, err + } + return false, nil +} + +func decodeCompletedTurnMessage(raw json.RawMessage) (completedTurnMessage, bool) { + var completed itemCompletedPayload + if err := json.Unmarshal(raw, &completed); err != nil || completed.Type != "item_completed" { + return completedTurnMessage{}, false + } + + var item turnItemMessagePayload + if err := json.Unmarshal(completed.Item, &item); err != nil { + return completedTurnMessage{}, false + } + + role := "" + switch item.Type { + case "UserMessage": + role = "user" + case "AgentMessage": + role = "assistant" + default: + return completedTurnMessage{}, false + } + + parts := make([]string, 0, len(item.Content)) + hasAttachment := false + for _, content := range item.Content { + switch strings.ToLower(content.Type) { + case "text": + if text := strings.TrimSpace(content.Text); text != "" { + parts = append(parts, text) + } + case "image", "local_image": + hasAttachment = true + case "audio", "local_audio": + hasAttachment = true + case "skill", "mention": + if name := strings.TrimSpace(content.Name); name != "" { + parts = append(parts, "["+name+"]") + } + } + } + + text := strings.Join(parts, "\n") + if text == "" && hasAttachment { + text = imageOnlyMessagePlaceholder + } + return completedTurnMessage{ID: item.ID, Role: role, Text: text}, true +} From b100c289dbfce7ebfe52f6c78ab9a6fc0509d9b0 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 03:16:55 -0700 Subject: [PATCH 02/13] feat(search): word matching, first-hit time, citations, parallel scan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dogfooding "when did we first mention semantica" exposed four defects at once: content matching was substring-only (47 hits, 46 of them "semantically", and no way to tell 0 real hits from 46), results carried no earliest-match time and ranked only by hit count, a cold scan took 6m18s on one core in total silence, and a summary hit short-circuited the content scan of the one session that mattered. - -w/--word: ASCII word boundary on whichever side of the query starts or ends with a word character, applied to names, summaries, conversation text and --raw lines. CJK queries are unaffected. - first_hit per content result: FIRST column, RFC3339 in --json, and --sort first|last|hits (hits stays the default). - --hits: one citation row per matching message — time, session, role, message id, quote — oldest first, capped by -n with a visible "showing N of M". Under --raw the unit is a transcript line anchored by its own uuid/type/timestamp. - Bounded worker pool (up to 8), non-allocating case-fold prefilter that stops at the first hit, stderr progress on a TTY: 1m14s -> 7.6s warm on a 3.5 GB store. - Summary hits keep their content evidence instead of short-circuiting. - -n shorthand for --limit on sessions, projects and log. Remaining open from the same dogfood: history.jsonl is still not a searchable source. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 13 + README.md | 2 + ...idence-citations-lessons-from-semantica.md | 73 ++ ...026-08-18-search-word-boundary-dogfood.org | 92 ++ internal/cmd/log.go | 2 +- internal/cmd/projects.go | 2 +- internal/cmd/search.go | 815 ++++++++++++++---- internal/cmd/search_test.go | 317 ++++++- internal/cmd/sessions.go | 2 +- skills/ccx/SKILL.md | 9 + 10 files changed, 1166 insertions(+), 161 deletions(-) create mode 100644 docs/design/0005-evidence-citations-lessons-from-semantica.md create mode 100644 docs/devlog/2026-08-18-search-word-boundary-dogfood.org diff --git a/CHANGELOG.md b/CHANGELOG.md index 03b6986..799eb99 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,19 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ## [Unreleased] +### Added +- **`ccx search -w/--word` matches whole words.** Matching was substring-only, so a term that prefixes a common word was unanswerable: `search --content semantica` returned 47 sessions, 46 of them "semantic*ally*", and ccx alone could not tell 0 real hits from 46 (docs/devlog/2026-08-18-search-word-boundary-dogfood.org). `-w` demands an ASCII word boundary on each side of the query that starts/ends with a word character (so "semantica-agi" and "(semantica)" still hit; CJK queries are unaffected) and applies to names, summaries, conversation text, and `--raw` lines alike. +- **`ccx search --hits` turns matches into citations.** One row per matching message — time, session, role, message id, quote — oldest first across sessions, `-n`-capped with a visible "showing N of M". The anchors are the same ones `trace` and `view` use, so a claim built on a search can point at its evidence (design: docs/design/0005-evidence-citations-lessons-from-semantica.md). Under `--raw` the unit is a transcript line, anchored by its own `uuid`/`type`/`timestamp`. +- **`ccx search --content` reports when: `FIRST` column, `first_hit` in `--json`, `--sort first|last|hits`.** "When did we first mention X" needs the earliest matching message and an oldest-first order; results only carried session end time and ranked by hit count. Each content hit now records the timestamp of its earliest matching message (parsed messages by default; the raw line's top-level `timestamp` under `--raw`), printed as `FIRST` and sortable with `--sort first`; `--sort last` orders by session activity; `--sort hits` is the old order and the default. + +### Changed +- **`ccx search --content` is ~10x faster and shows progress.** The scan ran on one core and lowercased every transcript line: 6m18s cold / 1m14s warm over a 3.5 GB store, silent throughout. Sessions now scan on a bounded worker pool (up to 8), the raw prefilter matches case-insensitively without allocating and stops at the first hit, and a `scanning N/M sessions` line ticks on stderr when it is a terminal. Same store, warm: 7.6s. + +### Fixed +- **`-n` is the `--limit` shorthand everywhere.** Only `search` had it; `sessions -n 2` failed with "unknown shorthand flag". `sessions`, `projects`, and `log` now accept `-n` too. +- **A session whose summary matched dropped its content evidence.** The summary hit short-circuited the content scan, so under `--content` the session where the term was actually discussed could be the one result with no hit count, previews, or first-hit time. Summary hits stay typed `session` but now carry the content fields. +- **Codex 0.147 conversations render again.** Codex moved its UI-facing user and assistant records from legacy `event_msg.user_message` / `agent_message` events to canonical `event_msg.item_completed` TurnItems. The Codex adapter now selects one conversation source per rollout, reads stable `UserMessage` / `AgentMessage` item IDs and content, keeps legacy rollouts working, and never mistakes raw `response_item` model input (which can contain injected instruction envelopes) for a human prompt. Discovery metadata, terminal view, web, export, search, and trace now agree; the parse-cache format is bumped so upgrades cannot serve blank cached sessions. + ## [0.15.0] - 2026-08-11 ### Added diff --git a/README.md b/README.md index 328958e..f83d1b7 100644 --- a/README.md +++ b/README.md @@ -80,6 +80,8 @@ ccx export --shape human # Only the human's turns, citable ccx trace [session] -o trace.json # Extract evidence for context folding ccx log --scope yesterday --tz +8 --all --json # Time-sliced log evidence ccx search "auth bug" # Search across sessions + memory +ccx search --content -w --sort first goose # Whole-word content hits, earliest first +ccx search --content -w --hits goose # Every mention, quoted + anchored (time, session, message id) ccx run ccx-recap --agent claude # Run a bundled skill via an agent CLI ccx fork abc123 # Fork session to current project ccx doctor # Check setup diff --git a/docs/design/0005-evidence-citations-lessons-from-semantica.md b/docs/design/0005-evidence-citations-lessons-from-semantica.md new file mode 100644 index 0000000..d077592 --- /dev/null +++ b/docs/design/0005-evidence-citations-lessons-from-semantica.md @@ -0,0 +1,73 @@ +# Design: Evidence citations — what ccx takes from Semantica + +**Created**: 2026-08-18 +**Status**: Accepted (principles + shipped items); Proposed (roadmap) +**CLI**: `ccx search --hits`, `ccx search -w / --sort / FIRST` +**Source studied**: `reference/semantica-agi/semantica` @ 5c2901a (cloned +2026-08-18, 8.7k stars, "Graph-Native Infrastructure for Context and +Accountable AI Systems") and the org profile README. + +## Why we looked + +Semantica and ccx answer the same question for different subjects: +*what was decided, why, and what evidence supports it*. Semantica does +it for enterprise AI decisions (a Python graph library: context graph, +decision records, W3C PROV-O provenance, causal chains, temporal +queries). ccx does it for coding-agent sessions (a Go CLI over the +transcripts Claude Code / Codex / Grok leave behind). The overlap is +the methodology, not the code. This note records which of their +principles we adopt, how each maps onto ccx, and what we reject. + +## Principles adopted + +| # | Semantica principle (where) | ccx mapping | Status | +|---|---|---|---| +| 1 | Explain the observable trail, not the model's cognition. "Semantica explains what's *outside* the model: the context fed in, the decision produced, its provenance, the execution trail" (`docs/concepts.md:20`). | Already ccx's stance: `trace` is "a factual record with zero interpretation; judgment belongs to the skills" (`internal/trace/types.go:10`). We keep the line hard: CLI = facts, skills = judgment. | held | +| 2 | Every fact carries a **verbatim quote plus a location** (`ProvenanceEntry.source_quote`, `source_location`, `semantica/provenance/schemas.py`). A decision without a quotable span is an assertion, not a finding. | Search results were counts per session. `--hits` makes each match a citation: time, session, role, message id, quote (`ccx search --content -w --hits X`). `trace` already anchors steps to `message_id`/`tool_id`. Rule: any ccx output that names a fact carries (session, message id, time). | shipped | +| 3 | Decision = **scenario / reasoning / outcome** as separate fields, plus who decided (`decision_models.py:87`). | Trace already carries the triple implicitly: `Turn.user_text` (scenario), `Step.narration` (reasoning at the moment of decision), `Step.mutations` + linked commits (outcome), `session.model` (decision maker). Make the mapping explicit in the recap/retro skill prompts; no schema change. | doc | +| 4 | **Supersedes ≠ derives-from** (`previous_version_id` vs `derived_from_id`); retraction is a tombstone, not a delete (`prov:Invalidation`). | `Turn.superseded` / `superseded_by_turn` already keeps the edited-away prompt as evidence. Missing: cross-session lineage (fork/resume) as a derives-from edge. | partial | +| 5 | **Bi-temporal**: when a fact was true vs when it was recorded (`kg/temporal_model.py`). | `search` now reports `FIRST` (when a term entered the record) beside `LAST` (session activity); `--sort first` answers "when did we first say X". `log --scope` slices by record time. | shipped | +| 6 | Truncation is explicit, never silent (`trace_decision_causality` appends `{"truncated": true, ...}`). | Already a ccx rule ("showing N of M (raise with -n)"); `--hits` follows it. | held | +| 7 | Discrete strength bands + one-sentence interpretation from **one shared threshold function** (`utils/helpers.py:586`), not invented decimals. | For skills: state claim strength as verified / claimed / inferred (ccx-recap already does); never emit a confidence float the data cannot support. | held | +| 8 | Symptom-named regression tests as readability contracts (`test_add_decision_scenario_stored_as_content`: humans got an opaque id where prose was expected). | Adopt the naming: e.g. `TestSessionSearcherSummaryHitKeepsContent` (this session) names the symptom, not the function. | adopted | + +## Rejected (for ccx) + +- Graph database / embeddings / vector precedent search. ccx is a + single static binary over files; lexical search with word + boundaries answers "was X ever discussed" exactly, and skills do + the semantic step. +- PROV-O / RDF export, policy engine, SHACL. Compliance theatre for our + use; the transcript file *is* the primary evidence and ccx is + read-only over it. +- Hash-chained records. Tamper evidence matters when the store is + yours; ccx does not own the store. +- God objects and alias APIs (three public names per behavior). Keep + one name per behavior; name by the user's question. +- Uncalibrated confidence floats multiplied through "decay". Ordering + is useful; the decimals are not. + +## Shipped in this pass (2026-08-18) + +- `search -w/--word`; `FIRST` column / `first_hit` / `--sort hits|first|last`; + `--hits` citations; summary hits keep content evidence; parallel scan + (~10x) with progress; `-n` shorthand on `sessions`/`projects`/`log`. + Devlog: `docs/devlog/2026-08-18-search-word-boundary-dogfood.org`. + +## Roadmap (small, ordered) + +1. `ccx log --match QUERY [-w]`: records containing a term inside a time + window — the bi-temporal slice (`search --hits` is unbounded in + time; `log` is bounded but cannot filter by term). Reuse + `textMatcher`. +2. `ccx view --at ` (or `--grep`): walk from a + citation to its surrounding context without leaving ccx (open since + `docs/devlog/2026-08-03-content-search-noise.org` finding 4). +3. `~/.claude/history.jsonl` as a `type: prompt` search source: the + longest-lived evidence (prompts back to 2025-09) for "when did we + first say X" once session files have been cleaned up. +4. Session lineage: `fork`/`resume` parents as derives-from edges in + `sessions --json` and `trace`, so a decision chain can cross + session boundaries. +5. Readability contract test for `trace`: every step carries + human-readable narration or a mutation summary, never only ids. diff --git a/docs/devlog/2026-08-18-search-word-boundary-dogfood.org b/docs/devlog/2026-08-18-search-word-boundary-dogfood.org new file mode 100644 index 0000000..8dcf0e0 --- /dev/null +++ b/docs/devlog/2026-08-18-search-word-boundary-dogfood.org @@ -0,0 +1,92 @@ +* [2026-08-18] Dev Log: Dogfood findings — "when did we mention X" via search :SEARCH: + +** Context +Question: "when did we first mention semantica / semantica-agi?" Answered +against the real host store (~/.claude 4411 jsonl / 3.5G, ~/.codex 1026, +~/.grok) with a from-source build (v0.15.0-dirty, main @ d3e00c9 + WIP +codex rollout drift). Answer: never before this session — every one of +ccx's 46 "semantica" content hits was the word "semantic*ally*". The +answer was findable, but only by cross-checking with =/usr/bin/grep -w=; +ccx alone could not distinguish 0 real hits from 46. Findings recorded, +not fixed. + +** Findings +1. Substring-only matching, no word-boundary or regex option. + =search --content semantica= returned 47 results, 46 of them + "semantically". There is no =-w= / =--regex= / =--exact= flag and + the phrase syntax gives no way to express a boundary ("semantica " + misses "semantica." and "semantica-agi"). For a project/product + name that is a prefix of a common English word, the tool cannot + answer "was this term ever used"; grep -w can. Repro: + =ccx search --content semantica -n 0 --json | jq length= -> 47; + =grep -rlwi semantica ~/.claude/projects= -> 1 (this session). +2. No way to sort or filter by first-mention time. Results rank by + Priority then hit count; LAST is session end. A "when did we first + say X" question needs the timestamp of the earliest matching + message and an oldest-first order. Neither exists; the top result + is whichever session repeated the term most. +3. Cold content scan is slow and silent: 6m18s wall (226s user, 168s + sys, 104% cpu) for one query on the bind-mounted store; warm re-run + 1m14s; grep -rli on the same tree ~48s wall / 5s user. Causes are + visible in internal/cmd/search.go: (a) the loop over sessions is + sequential — zero goroutines — so a 10-core box uses one; (b) + =countMatchingLines= allocates =strings.ToLower(line)= for every + line of every file (3.5G of allocations); (c) the prefiltered + candidates then get a full =ParseSession=. Nothing is printed until + the whole scan finishes, so a 6-minute run looks hung (the harness + backgrounded it at 300s). At minimum: a stderr progress line + ("scanned 1200/4411 sessions, 12 candidates") and results streamed + as found; ideally a worker pool + a non-allocating case-fold match. +4. Content search only covers session files; =~/.claude/history.jsonl= + (every prompt since 2025-09-28, older than any surviving session + file at 2025-12-07) is not searched. For "when did we mention X" + questions the prompt history is the longest-lived evidence and the + cheapest to scan (8 MB); today it needs raw grep. +5. Small: the current session appears typed =session= (summary match, + Priority 1) and is therefore never scanned for content, so its + content hits and previews are missing from =--json= even though it + is the only real match. When a session matches both summary and + content, the content evidence should still be attached. + +** Fix directions +- =--word= (or make it the default for single-token queries with an + escape hatch) and =--regex=; both compose with =--content= / =--raw=. + Prefilter stays substring; boundary/regex applies at count time. +- Per-result =first_hit= timestamp (message time of the earliest + match) plus =--sort first|last|hits= and =--oldest=; print FIRST + alongside LAST when =--content= is on. +- Parallel scan across sessions (bounded worker pool), case-fold + without per-line allocation, progress on stderr, stream results. +- Include =history.jsonl= as a searchable source (=type: prompt=, + carries project + timestamp). +- Attach content previews to session-summary matches when =--content= + is on. + +** Resolution [2026-08-18] +Findings 1, 2, 3, 5 fixed in the same session (internal/cmd/search.go); +finding 4 (history.jsonl as a source) stays open. +- =FEAT= =-w/--word=: textMatcher with ASCII =\b= on the word-char sides + of the query, applied to names, summaries, conversation text, and + =--raw= lines. Live: =search --content -w semantica= -> 1 result + (this session) vs 47 without =-w=; agrees with =grep -rlwi=. +- =FEAT= first-hit time per content result: =FIRST= column, =first_hit= + (RFC3339) in =--json=, =--sort hits|first|last=. Verified: first_hit + 2026-08-18T07:48:10Z == earliest raw line with the word. +- =PERF= bounded worker pool (min(NumCPU, 8)), non-allocating + case-fold line match, prefilter stops at first hit, stderr progress + when a TTY. Warm scan 1m14s -> 7.6s on the same 3.5 GB store. +- =FIX= summary hits keep content evidence (matches, previews, + first_hit) instead of short-circuiting the scan. +- Tests: TestTextMatcherWord, TestIndexFold, TestLineTimestamp, + TestSessionSearcherSummaryHitKeepsContent, TestSessionSearcherMatchAllOrder, + TestSortSearchResults; existing scan tests updated to the new + signatures. +- =FEAT= (after studying semantica-agi/semantica; see + docs/design/0005-evidence-citations-lessons-from-semantica.md) + =--hits=: every matching message as a citation — time, session, + role, message id, quote — oldest first; =--raw --hits= per line. + Live: =search --content -w --hits semantica= -> 11 rows, first + c8bd2144 @ 07:48:10Z, matching the raw file. Test: + TestSessionSearcherHits. +- =FIX= =-n= shorthand for =--limit= on sessions/projects/log + (=sessions -n 2= failed "unknown shorthand flag"). diff --git a/internal/cmd/log.go b/internal/cmd/log.go index 5d5e0e3..9e0ebcd 100644 --- a/internal/cmd/log.go +++ b/internal/cmd/log.go @@ -55,7 +55,7 @@ func init() { logCmd.Flags().BoolVar(&logRaw, "raw", false, "include raw JSONL lines in JSON output") logCmd.Flags().BoolVar(&logAll, "all", false, "slice logs across all projects") logCmd.Flags().StringVarP(&logProvider, "provider", "p", "", "filter by provider: cc, cx, all") - logCmd.Flags().IntVar(&logLimit, "limit", 0, "limit records in JSON output (0 = no limit)") + logCmd.Flags().IntVarP(&logLimit, "limit", "n", 0, "limit records in JSON output (0 = no limit)") } func runLog(cmd *cobra.Command, args []string) error { diff --git a/internal/cmd/projects.go b/internal/cmd/projects.go index 2f08306..87aeb45 100644 --- a/internal/cmd/projects.go +++ b/internal/cmd/projects.go @@ -29,7 +29,7 @@ var ( func init() { projectsCmd.Flags().StringVar(&projectsSort, "sort", "time", "sort by: name, time, sessions") - projectsCmd.Flags().IntVar(&projectsLimit, "limit", 0, "limit number of projects (0 = no limit)") + projectsCmd.Flags().IntVarP(&projectsLimit, "limit", "n", 0, "limit number of projects (0 = no limit)") projectsCmd.Flags().BoolVar(&projectsJSON, "json", false, "output as JSON") } diff --git a/internal/cmd/search.go b/internal/cmd/search.go index 7966a8b..90a4a97 100644 --- a/internal/cmd/search.go +++ b/internal/cmd/search.go @@ -7,9 +7,13 @@ import ( "io" "os" "path/filepath" + "regexp" + "runtime" "sort" "strings" + "sync" "text/tabwriter" + "time" "unicode/utf8" "github.com/spf13/cobra" @@ -26,24 +30,41 @@ var searchCmd = &cobra.Command{ The query is one case-insensitive phrase: multiple words must appear adjacent and in order ("fix bug" won't match "bug ... fix"). For -term-level matching, run one search per term. Exits 0 either way; -zero matches just prints "No results found." +term-level matching, run one search per term. Matching is substring +by default; -w/--word requires whole words, so "semantica" no longer +matches "semantically". Exits 0 either way; zero matches just prints +"No results found." With --content, also scan conversation text inside session files (including subagent files): user prompts and assistant replies, -ranked by hit count with a matched-text preview. Injected noise — -tool results, hook attachments, command echoes — doesn't count. +ranked by hit count with a matched-text preview and the time of the +earliest match (FIRST). Injected noise — tool results, hook +attachments, command echoes — doesn't count. Add --raw to match every raw transcript line instead: grep parity, no parse, misses nothing grep would find. +--sort orders results: hits (default: match kind, then hit count), +first (earliest match first — "when did we first mention X"), last +(most recently active session first). + +--hits lists every matching message instead of one row per session: +time, session, role, message id, and the quote around the match, +oldest first. Each row is a citation — the same anchors ccx trace +and view use — so a claim built on a search can point at its +evidence. -n caps the rows. + Examples: ccx search auth # Find sessions about authentication ccx search myproject # Find project by name ccx search "fix bug" # Phrase match: adjacent words, in order ccx search -t session # Only search sessions ccx search --content goose # Scan conversation text (slower) - ccx search --raw goose # Grep parity over raw lines`, + ccx search --raw goose # Grep parity over raw lines + ccx search --content -w --sort first semantica + # Whole-word, earliest mention first + ccx search --content -w --hits semantica + # Every mention, quoted and anchored`, Args: cobra.MinimumNArgs(1), RunE: runSearch, } @@ -58,6 +79,9 @@ var ( searchModel string searchContent bool searchRaw bool + searchWord bool + searchSort string + searchHits bool ) func init() { @@ -70,6 +94,9 @@ func init() { searchCmd.Flags().StringVar(&searchModel, "model", "", "filter by model name substring") searchCmd.Flags().BoolVar(&searchContent, "content", false, "also scan conversation text in session files (slower)") searchCmd.Flags().BoolVar(&searchRaw, "raw", false, "content scan matches every raw transcript line (grep parity; implies --content)") + searchCmd.Flags().BoolVarP(&searchWord, "word", "w", false, "match whole words only (\"semantica\" won't match \"semantically\")") + searchCmd.Flags().StringVar(&searchSort, "sort", "hits", "order results: hits, first, last") + searchCmd.Flags().BoolVar(&searchHits, "hits", false, "list every matching message with time, role, message id, and quote (implies --content)") rootCmd.AddCommand(searchCmd) } @@ -82,8 +109,13 @@ type searchResult struct { Summary string `json:"summary"` Time string `json:"time,omitempty"` Matches int `json:"matches,omitempty"` + FirstHit string `json:"first_hit,omitempty"` // RFC3339 time of the earliest match Previews []contentPreview `json:"previews,omitempty"` Priority int `json:"-"` + + firstHit time.Time + lastTime time.Time + hits []searchHit } // contentPreview is one matched conversation snippet, role-labeled so @@ -93,15 +125,151 @@ type contentPreview struct { Text string `json:"text"` } +// searchHit is one matching message: a quote plus the anchors needed +// to walk back to it (session file, message id, time). Under --raw the +// unit is a transcript line and Role is the line's top-level type. +type searchHit struct { + Project string `json:"project"` + Session string `json:"session"` + Path string `json:"path"` + MessageID string `json:"message_id,omitempty"` + Time time.Time `json:"time"` + Role string `json:"role"` + Matches int `json:"matches"` // occurrences within this message + Quote string `json:"quote"` +} + const maxContentPreviews = 3 +// textMatcher decides how QUERY matches text. Default: case-insensitive +// substring. Word mode adds ASCII word boundaries on each side of the +// query that starts/ends with a word character, so "semantica" no +// longer matches "semantically" while "中文" still matches inside a CJK +// run (no \w on either side, so no boundary is demanded there). +type textMatcher struct { + query string // lowercase literal + re *regexp.Regexp // nil in substring mode +} + +func newTextMatcher(query string, word bool) textMatcher { + m := textMatcher{query: strings.ToLower(query)} + if !word || m.query == "" { + return m + } + pat := regexp.QuoteMeta(m.query) + if isWordByte(m.query[0]) { + pat = `\b` + pat + } + if isWordByte(m.query[len(m.query)-1]) { + pat += `\b` + } + m.re = regexp.MustCompile(`(?i)` + pat) + return m +} + +func isWordByte(c byte) bool { + return c == '_' || (c >= '0' && c <= '9') || (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') +} + +// literal is the substring matcher for the same query: a superset of +// word matches, so it is a valid cheap prefilter. +func (m textMatcher) literal() textMatcher { return textMatcher{query: m.query} } + +// index returns the byte offset and length of the first match in +// text, or -1, 0. +func (m textMatcher) index(text string) (int, int) { + if m.re == nil { + return indexFold(text, m.query), len(m.query) + } + loc := m.re.FindStringIndex(text) + if loc == nil { + return -1, 0 + } + return loc[0], loc[1] - loc[0] +} + +func (m textMatcher) matches(text string) bool { + i, _ := m.index(text) + return i >= 0 +} + +// count returns the number of non-overlapping matches in text. +func (m textMatcher) count(text string) int { + if m.re == nil { + return countFold(text, m.query) + } + return len(m.re.FindAllStringIndex(text, -1)) +} + +// indexFold is a case-insensitive strings.Index for a lowercase query +// that does not allocate a lowered copy of s: the raw scan runs it +// over every transcript line, and lowering gigabytes was the scan's +// dominant cost. Non-ASCII queries fall back to the lowered copy +// (Unicode case folding is not byte-stable). +func indexFold(s, query string) int { + if query == "" { + return 0 + } + if !isASCII(query) { + return strings.Index(strings.ToLower(s), query) + } + n := len(query) + first := query[0] + for i := 0; i+n <= len(s); i++ { + c := s[i] + if c >= 'A' && c <= 'Z' { + c += 'a' - 'A' + } + if c != first { + continue + } + if strings.EqualFold(s[i:i+n], query) { + return i + } + } + return -1 +} + +func countFold(s, query string) int { + if query == "" { + return 0 + } + if !isASCII(query) { + return strings.Count(strings.ToLower(s), query) + } + n := 0 + for { + i := indexFold(s, query) + if i < 0 { + return n + } + n++ + s = s[i+len(query):] + } +} + +func isASCII(s string) bool { + for i := 0; i < len(s); i++ { + if s[i] >= utf8.RuneSelf { + return false + } + } + return true +} + func runSearch(cmd *cobra.Command, args []string) error { - query := strings.ToLower(strings.Join(args, " ")) + m := newTextMatcher(strings.Join(args, " "), searchWord) + query := m.query backend := provider.Default() - if searchRaw { + if searchRaw || searchHits { searchContent = true } + switch searchSort { + case "hits", "first", "last": + default: + return fmt.Errorf("invalid --sort %q: want hits, first, or last", searchSort) + } after, err := config.ParseDate(searchAfter) if err != nil { @@ -124,6 +292,7 @@ func runSearch(cmd *cobra.Command, args []string) error { } var results []searchResult + var candidates []sessionCandidate for _, p := range projects { // Backends set Name to the human-readable form; the encoding @@ -140,9 +309,7 @@ func runSearch(cmd *cobra.Command, args []string) error { // Project name match (skip if filtering to sessions only) if searchType != "session" { - nameMatch := strings.Contains(strings.ToLower(p.EncodedName), query) || - strings.Contains(strings.ToLower(projPath), query) || - strings.Contains(strings.ToLower(projDisplay), query) + nameMatch := m.matches(p.EncodedName) || m.matches(projPath) || m.matches(projDisplay) providerMatch := filter.Provider == "" if !providerMatch { @@ -173,100 +340,31 @@ func runSearch(cmd *cobra.Command, args []string) error { if !filter.IsEmpty() && !filter.Match(s) { continue } + candidates = append(candidates, sessionCandidate{session: s, project: projDisplay}) + } + } - // Session ID match (high priority) - if strings.HasPrefix(strings.ToLower(s.ID), query) { - results = append(results, searchResult{ - Type: "session", - Project: projDisplay, - Session: truncateID(s.ID, 8), - Path: s.FilePath, - Summary: sessionSummaryPreview(s.Summary, 64), - Time: formatAge(s.EndTime), - Priority: 0, - }) - continue - } - - // Summary match - if strings.Contains(strings.ToLower(s.Summary), query) { - results = append(results, searchResult{ - Type: "session", - Project: projDisplay, - Session: truncateID(s.ID, 8), - Path: s.FilePath, - Summary: sessionSummaryPreview(s.Summary, 64), - Time: formatAge(s.EndTime), - Priority: 2, - }) - continue - } - - // Content scan. The default counts conversation text only — - // user prompts and assistant replies — so ranking follows - // discussion, not injected boilerplate (a hook line fired - // every turn once outranked the real answer 327 hits to 13; - // docs/devlog/2026-08-03-content-search-noise.org). --raw - // keeps grep parity over raw transcript lines, main file - // plus subagent files: no parse, works for every provider's - // format, misses nothing grep would find. - if searchContent { - if searchRaw { - if n := countContentMatches(s.FilePath, query); n > 0 { - results = append(results, searchResult{ - Type: "content", - Project: projDisplay, - Session: truncateID(s.ID, 8), - Path: s.FilePath, - Summary: fmt.Sprintf("%d hits · %s", n, sessionSummaryPreview(s.Summary, 48)), - Time: formatAge(s.EndTime), - Matches: n, - Priority: 3, - }) - } - continue - } + searcher := sessionSearcher{m: m, content: searchContent, raw: searchRaw, hits: searchHits, parser: backend} + for _, r := range searcher.matchAll(candidates, searchWorkers(searchContent)) { + results = append(results, *r) + } - // Cheap line-scan prefilter before the full parse; only - // trustworthy when JSON escaping can't hide the query. - if rawPrefilterSafe(query) && countContentMatches(s.FilePath, query) == 0 { - continue - } - n, previews := scanConversationText(backend, s.FilePath, query) - if n == 0 { - continue - } - summary := fmt.Sprintf("%d hits · %s", n, sessionSummaryPreview(s.Summary, 48)) - if len(previews) > 0 { - summary = fmt.Sprintf("%d hits · [%s] %s", n, previews[0].Role, truncateDisplay(previews[0].Text, 56)) - } - results = append(results, searchResult{ - Type: "content", - Project: projDisplay, - Session: truncateID(s.ID, 8), - Path: s.FilePath, - Summary: summary, - Time: formatAge(s.EndTime), - Matches: n, - Previews: previews, - Priority: 3, - }) - } - } + if searchHits { + return printSearchHits(collectHits(results)) } // Search memory files if searchType != "project" && searchType != "session" { settings := config.Load() for _, home := range []string{settings.ClaudeHome, settings.CodexHome} { - searchMemoryDir(home, "projects", query, filter, &results) + searchMemoryDir(home, "projects", m, filter, &results) // Global files for _, name := range []string{"CLAUDE.md", "instructions.md", "AGENTS.md"} { path := filepath.Join(home, name) if _, err := os.Stat(path); err != nil { continue } - if strings.Contains(strings.ToLower(name), query) { + if m.matches(name) { results = append(results, searchResult{ Type: "memory", Project: filepath.Base(home), @@ -279,14 +377,7 @@ func runSearch(cmd *cobra.Command, args []string) error { } } - // Sort by priority, then by match count within content results. - // Stable so equal-rank results keep discovery order across runs. - sort.SliceStable(results, func(i, j int) bool { - if results[i].Priority != results[j].Priority { - return results[i].Priority < results[j].Priority - } - return results[i].Matches > results[j].Matches - }) + sortSearchResults(results, searchSort) // Limit results — never silently. if searchLimit > 0 && len(results) > searchLimit { @@ -311,33 +402,98 @@ func runSearch(cmd *cobra.Command, args []string) error { return enc.Encode(results) } - return printSearchResults(results) + return printSearchResults(results, searchContent) +} + +// sortSearchResults orders results. "hits": match kind (Priority), +// then hit count — stable, so equal-rank results keep discovery order +// across runs. "first": earliest match first; results with no match +// time (projects, memory, name-only hits) trail in hits order. +// "last": most recently active session first, timeless trailing. +func sortSearchResults(results []searchResult, mode string) { + byHits := func(a, b searchResult) bool { + if a.Priority != b.Priority { + return a.Priority < b.Priority + } + return a.Matches > b.Matches + } + sort.SliceStable(results, func(i, j int) bool { + a, b := results[i], results[j] + switch mode { + case "first": + switch { + case a.firstHit.IsZero() != b.firstHit.IsZero(): + return !a.firstHit.IsZero() + case !a.firstHit.IsZero() && !a.firstHit.Equal(b.firstHit): + return a.firstHit.Before(b.firstHit) + } + case "last": + switch { + case a.lastTime.IsZero() != b.lastTime.IsZero(): + return !a.lastTime.IsZero() + case !a.lastTime.IsZero() && !a.lastTime.Equal(b.lastTime): + return a.lastTime.After(b.lastTime) + } + } + return byHits(a, b) + }) } -func printSearchResults(results []searchResult) error { +// searchWorkers sizes the content-scan pool. The scan is I/O plus +// per-line matching; one goroutine per CPU (capped) keeps a cold +// store busy without thrashing it. +func searchWorkers(content bool) int { + if !content { + return 1 + } + n := runtime.NumCPU() + if n > 8 { + n = 8 + } + if n < 1 { + n = 1 + } + return n +} + +func printSearchResults(results []searchResult, withFirst bool) error { w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0) // LAST = last activity (session end time): the same timestamp // --after/--before filter on and results sort by, so a filtered - // row never displays a date outside the requested window. - fmt.Fprintln(w, "TYPE\tPROJECT\tSESSION\tSUMMARY\tLAST") + // row never displays a date outside the requested window. FIRST = + // earliest matching message, shown when content was scanned. + if withFirst { + fmt.Fprintln(w, "TYPE\tPROJECT\tSESSION\tSUMMARY\tFIRST\tLAST") + } else { + fmt.Fprintln(w, "TYPE\tPROJECT\tSESSION\tSUMMARY\tLAST") + } for _, r := range results { session := r.Session if session == "" { session = "-" } - time := r.Time - if time == "" { - time = "-" + last := r.Time + if last == "" { + last = "-" + } + if withFirst { + first := "-" + if !r.firstHit.IsZero() { + first = r.firstHit.Local().Format("2006-01-02 15:04") + } + fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", + r.Type, truncateDisplay(cleanDisplayText(r.Project), 24), session, cleanDisplayText(r.Summary), first, last) + continue } fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", - r.Type, truncateDisplay(cleanDisplayText(r.Project), 24), session, cleanDisplayText(r.Summary), time) + r.Type, truncateDisplay(cleanDisplayText(r.Project), 24), session, cleanDisplayText(r.Summary), last) } return w.Flush() } -func searchMemoryDir(home, subdir, query string, filter config.SessionFilter, results *[]searchResult) { +func searchMemoryDir(home, subdir string, m textMatcher, filter config.SessionFilter, results *[]searchResult) { projectsDir := filepath.Join(home, subdir) projEntries, err := os.ReadDir(projectsDir) if err != nil { @@ -357,7 +513,7 @@ func searchMemoryDir(home, subdir, query string, filter config.SessionFilter, re if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".md") { continue } - if strings.Contains(strings.ToLower(entry.Name()), query) { + if m.matches(entry.Name()) { *results = append(*results, searchResult{ Type: "memory", Project: projDisplay, @@ -383,57 +539,329 @@ type sessionParser interface { ParseSession(filePath string) (*parser.Session, error) } +// sessionCandidate is one session that passed the provider/date/model +// filters and still has to be matched against the query. +type sessionCandidate struct { + session *parser.Session + project string +} + +// sessionSearcher matches one session against the query: ID prefix, +// summary, and — with content on — conversation text or raw lines. +type sessionSearcher struct { + m textMatcher + content bool + raw bool + hits bool // collect every matching message, not just previews + parser sessionParser +} + +// matchAll runs match over candidates on `workers` goroutines and +// returns the hits in candidate order (so equal-rank results keep +// discovery order). Progress goes to stderr when it is a terminal. +func (ss sessionSearcher) matchAll(candidates []sessionCandidate, workers int) []*searchResult { + out := make([]*searchResult, len(candidates)) + if len(candidates) == 0 { + return nil + } + if workers < 1 { + workers = 1 + } + if workers > len(candidates) { + workers = len(candidates) + } + + progress := newScanProgress(len(candidates), ss.content) + var wg sync.WaitGroup + next := make(chan int) + for w := 0; w < workers; w++ { + wg.Add(1) + go func() { + defer wg.Done() + for i := range next { + out[i] = ss.match(candidates[i].session, candidates[i].project) + progress.tick() + } + }() + } + for i := range candidates { + next <- i + } + close(next) + wg.Wait() + progress.done() + + hits := out[:0] + for _, r := range out { + if r != nil { + hits = append(hits, r) + } + } + return hits +} + +// match returns the result for one session, or nil. A summary hit +// stays typed "session" but still carries content evidence (matches, +// previews, first hit) when content is on — the session where the +// term was actually discussed must not be the one result without +// its discussion. +func (ss sessionSearcher) match(s *parser.Session, projDisplay string) *searchResult { + // Session ID match (high priority) + if strings.HasPrefix(strings.ToLower(s.ID), ss.m.query) { + return &searchResult{ + Type: "session", + Project: projDisplay, + Session: truncateID(s.ID, 8), + Path: s.FilePath, + Summary: sessionSummaryPreview(s.Summary, 64), + Time: formatAge(s.EndTime), + Priority: 0, + lastTime: s.EndTime, + } + } + + var res *searchResult + if ss.m.matches(s.Summary) { + res = &searchResult{ + Type: "session", + Project: projDisplay, + Session: truncateID(s.ID, 8), + Path: s.FilePath, + Summary: sessionSummaryPreview(s.Summary, 64), + Time: formatAge(s.EndTime), + Priority: 2, + lastTime: s.EndTime, + } + } + if !ss.content { + return res + } + + // Content scan. The default counts conversation text only — + // user prompts and assistant replies — so ranking follows + // discussion, not injected boilerplate (a hook line fired + // every turn once outranked the real answer 327 hits to 13; + // docs/devlog/2026-08-03-content-search-noise.org). --raw + // keeps grep parity over raw transcript lines, main file + // plus subagent files: no parse, works for every provider's + // format, misses nothing grep would find. + var ( + n int + first time.Time + previews []contentPreview + hits []searchHit + ) + if ss.raw { + var lines []rawHit + n, first, lines = countContentMatches(s.FilePath, ss.m, ss.hits) + for _, l := range lines { + hits = append(hits, searchHit{MessageID: l.id, Time: l.time, Role: l.role, Matches: 1, Quote: l.quote}) + } + } else { + // Cheap line-scan prefilter before the full parse; only + // trustworthy when JSON escaping can't hide the query. The + // literal matcher is a superset of word matches, so it + // prefilters both modes. + if rawPrefilterSafe(ss.m.query) && !sessionHasRawMatch(s.FilePath, ss.m.literal()) { + return res + } + n, first, previews, hits = scanConversationText(ss.parser, s.FilePath, ss.m, ss.hits) + } + if n == 0 { + return res + } + for i := range hits { + hits[i].Project = projDisplay + hits[i].Session = truncateID(s.ID, 8) + hits[i].Path = s.FilePath + } + + summary := fmt.Sprintf("%d hits · %s", n, sessionSummaryPreview(s.Summary, 48)) + if len(previews) > 0 { + summary = fmt.Sprintf("%d hits · [%s] %s", n, previews[0].Role, truncateDisplay(previews[0].Text, 56)) + } + if res == nil { + res = &searchResult{ + Type: "content", + Project: projDisplay, + Session: truncateID(s.ID, 8), + Path: s.FilePath, + Summary: summary, + Time: formatAge(s.EndTime), + Priority: 3, + lastTime: s.EndTime, + } + } + res.Matches = n + res.Previews = previews + res.firstHit = first + res.hits = hits + if !first.IsZero() { + res.FirstHit = first.UTC().Format(time.RFC3339) + } + return res +} + +// collectHits flattens per-session hits into one timeline, oldest +// first; hits without a timestamp trail in discovery order. +func collectHits(results []searchResult) []searchHit { + var hits []searchHit + for _, r := range results { + hits = append(hits, r.hits...) + } + sort.SliceStable(hits, func(i, j int) bool { + a, b := hits[i].Time, hits[j].Time + if a.IsZero() != b.IsZero() { + return !a.IsZero() + } + return a.Before(b) + }) + return hits +} + +func printSearchHits(hits []searchHit) error { + if searchLimit > 0 && len(hits) > searchLimit { + fmt.Fprintf(os.Stderr, "showing %d of %d hits (raise with -n)\n", searchLimit, len(hits)) + hits = hits[:searchLimit] + } + if len(hits) == 0 { + fmt.Println("No hits found.") + return nil + } + if searchJSON { + enc := json.NewEncoder(os.Stdout) + enc.SetIndent("", " ") + return enc.Encode(hits) + } + w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0) + fmt.Fprintln(w, "TIME\tSESSION\tROLE\tMESSAGE\tQUOTE") + for _, h := range hits { + t := "-" + if !h.Time.IsZero() { + t = h.Time.Local().Format("2006-01-02 15:04") + } + id := h.MessageID + if id == "" { + id = "-" + } + fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", t, h.Session, h.Role, truncateID(id, 8), cleanDisplayText(h.Quote)) + } + return w.Flush() +} + +// scanProgress reports content-scan progress on stderr when stderr is +// a terminal, so a multi-minute cold scan is visibly alive. Silent +// when piped: agents and scripts read stdout, and a stream of \r +// lines is noise there. +type scanProgress struct { + mu sync.Mutex + total int + done_ int + last time.Time + on bool +} + +func newScanProgress(total int, on bool) *scanProgress { + if on { + info, err := os.Stderr.Stat() + on = err == nil && info.Mode()&os.ModeCharDevice != 0 + } + return &scanProgress{total: total, on: on} +} + +func (p *scanProgress) tick() { + if !p.on { + return + } + p.mu.Lock() + defer p.mu.Unlock() + p.done_++ + if now := time.Now(); now.Sub(p.last) >= 200*time.Millisecond { + p.last = now + fmt.Fprintf(os.Stderr, "\rscanning %d/%d sessions", p.done_, p.total) + } +} + +func (p *scanProgress) done() { + if !p.on { + return + } + p.mu.Lock() + defer p.mu.Unlock() + fmt.Fprintf(os.Stderr, "\r%*s\r", len(fmt.Sprintf("scanning %d/%d sessions", p.total, p.total)), "") +} + // scanConversationText parses one session (the parser loads sidechain // files too) and searches only conversation text: text and thinking // blocks of user prompts and assistant messages. Tool results, hook // attachments, command echoes, and meta lines never count — that's -// what --raw is for. Returns total occurrences plus up to -// maxContentPreviews role-labeled snippets around the earliest -// matches. query must already be lowercase. -func scanConversationText(p sessionParser, path, query string) (int, []contentPreview) { +// what --raw is for. Returns total occurrences, the timestamp of the +// earliest matching message, and up to maxContentPreviews role-labeled +// snippets around the earliest matches. +func scanConversationText(p sessionParser, path string, m textMatcher, wantHits bool) (int, time.Time, []contentPreview, []searchHit) { sess, err := p.ParseSession(path) if err != nil { fmt.Fprintf(os.Stderr, "warning: skipping unparseable %s: %v\n", filepath.Base(path), err) - return 0, nil + return 0, time.Time{}, nil, nil } count := 0 + var first time.Time var previews []contentPreview + var hits []searchHit var walk func(msgs []*parser.Message) walk = func(msgs []*parser.Message) { - for _, m := range msgs { - if m.Kind == parser.KindUserPrompt || m.Kind == parser.KindAssistant { - role := m.Type - if m.IsSidechain { + for _, msg := range msgs { + if msg.Kind == parser.KindUserPrompt || msg.Kind == parser.KindAssistant { + role := msg.Type + if msg.IsSidechain { role = "agent" } - for _, b := range m.Content { + msgHits := 0 + quote := "" + for _, b := range msg.Content { if b.Type != "text" && b.Type != "thinking" { continue } - lower := strings.ToLower(b.Text) - n := strings.Count(lower, query) + n := m.count(b.Text) if n == 0 { continue } count += n - if len(previews) < maxContentPreviews { - previews = append(previews, contentPreview{ - Role: role, - Text: matchSnippet(b.Text, strings.Index(lower, query), len(query)), - }) + msgHits += n + if !msg.Timestamp.IsZero() && (first.IsZero() || msg.Timestamp.Before(first)) { + first = msg.Timestamp + } + if len(previews) < maxContentPreviews || (wantHits && quote == "") { + idx, qlen := m.index(b.Text) + snippet := matchSnippet(b.Text, idx, qlen) + if quote == "" { + quote = snippet + } + if len(previews) < maxContentPreviews { + previews = append(previews, contentPreview{Role: role, Text: snippet}) + } } } + if wantHits && msgHits > 0 { + hits = append(hits, searchHit{ + MessageID: msg.UUID, + Time: msg.Timestamp, + Role: role, + Matches: msgHits, + Quote: quote, + }) + } } - walk(m.Children) + walk(msg.Children) } } walk(sess.RootMessages) - return count, previews + return count, first, previews, hits } // matchSnippet cuts a display window around a match, clamped to rune -// boundaries. idx indexes the lowered copy of text; byte positions can +// boundaries. idx may index a lowered copy of text; byte positions can // drift on the rare rune whose lowercase form changes width, so bounds // are clamped rather than trusted. func matchSnippet(text string, idx, qlen int) string { @@ -480,46 +908,143 @@ func rawPrefilterSafe(query string) bool { return true } -// countContentMatches counts transcript lines containing query across -// the main session file and any subagent files beside it (layout +// sessionHasRawMatch reports whether any raw transcript line of the +// session — main file or subagent files — matches. Stops at the first +// hit: it is a prefilter, not a count. +func sessionHasRawMatch(sessionPath string, m textMatcher) bool { + if sessionPath == "" { + return false + } + if n, _, _ := scanRawLines(sessionPath, m, true, false); n > 0 { + return true + } + for _, f := range parser.SubagentFiles(sessionPath) { + if n, _, _ := scanRawLines(f, m, true, false); n > 0 { + return true + } + } + return false +} + +// countContentMatches counts transcript lines matching m across the +// main session file and any subagent files beside it (layout // knowledge lives in parser.SubagentFiles; providers without subagent -// files simply contribute none). -func countContentMatches(sessionPath, query string) int { +// files simply contribute none), and returns the earliest timestamp +// among matching lines. +func countContentMatches(sessionPath string, m textMatcher, wantHits bool) (int, time.Time, []rawHit) { if sessionPath == "" { - return 0 + return 0, time.Time{}, nil } - count := countMatchingLines(sessionPath, query) + count, first, hits := scanRawLines(sessionPath, m, false, wantHits) for _, f := range parser.SubagentFiles(sessionPath) { - count += countMatchingLines(f, query) + n, t, h := scanRawLines(f, m, false, wantHits) + count += n + first = earlier(first, t) + hits = append(hits, h...) } - return count + return count, first, hits } // countMatchingLines streams one JSONL file and counts lines matching -// query case-insensitively. bufio.Reader, not Scanner: transcript -// lines carrying embedded images exceed any fixed budget, and a -// silent early stop is exactly the false-negative class --content -// exists to kill. Unreadable files warn instead of lying "0 hits". -func countMatchingLines(path, query string) int { +// m, returning the earliest timestamp among them. +func countMatchingLines(path string, m textMatcher) (int, time.Time) { + n, first, _ := scanRawLines(path, m, false, false) + return n, first +} + +// rawHit is one matching raw transcript line: its top-level anchors +// plus a quote around the match. +type rawHit struct { + id string + role string + time time.Time + quote string +} + +// scanRawLines streams one JSONL file and counts lines matching m +// (stopping at the first when stopAtFirst). bufio.Reader, not +// Scanner: transcript lines carrying embedded images exceed any fixed +// budget, and a silent early stop is exactly the false-negative class +// --content exists to kill. Unreadable files warn instead of lying +// "0 hits". The earliest top-level "timestamp" among matching lines +// is returned when counting (every provider stamps its lines). +func scanRawLines(path string, m textMatcher, stopAtFirst, wantHits bool) (int, time.Time, []rawHit) { file, err := os.Open(path) if err != nil { fmt.Fprintf(os.Stderr, "warning: skipping unreadable %s: %v\n", filepath.Base(path), err) - return 0 + return 0, time.Time{}, nil } defer file.Close() count := 0 + var first time.Time + var hits []rawHit reader := bufio.NewReaderSize(file, 64*1024) for { line, err := reader.ReadString('\n') - if line != "" && strings.Contains(strings.ToLower(line), query) { - count++ + if line != "" { + if idx, qlen := m.index(line); idx >= 0 { + count++ + if stopAtFirst { + return count, first, nil + } + head := lineHead(line) + first = earlier(first, head.time) + if wantHits { + hits = append(hits, rawHit{id: head.id, role: head.role, time: head.time, quote: matchSnippet(line, idx, qlen)}) + } + } } if err != nil { if err != io.EOF { fmt.Fprintf(os.Stderr, "warning: read error in %s: %v\n", filepath.Base(path), err) } - return count + return count, first, hits } } } + +// lineHead is the top-level anchor set of one transcript line. +type lineHead_ struct { + id string + role string + time time.Time +} + +// lineHead extracts the top-level "timestamp", "uuid", and "type" of +// one transcript line; zero/empty when absent or unparseable. A full +// decode, not a substring hunt: tool results embed other objects' +// timestamps. +func lineHead(line string) lineHead_ { + var head struct { + Timestamp string `json:"timestamp"` + UUID string `json:"uuid"` + Type string `json:"type"` + } + if err := json.Unmarshal([]byte(line), &head); err != nil { + return lineHead_{} + } + out := lineHead_{id: head.UUID, role: head.Type} + for _, layout := range []string{time.RFC3339Nano, time.RFC3339} { + if t, err := time.Parse(layout, head.Timestamp); err == nil { + out.time = t + break + } + } + return out +} + +func lineTimestamp(line string) time.Time { return lineHead(line).time } + +// earlier returns the earlier of two times, ignoring zero values. +func earlier(a, b time.Time) time.Time { + switch { + case a.IsZero(): + return b + case b.IsZero(): + return a + case b.Before(a): + return b + } + return a +} diff --git a/internal/cmd/search_test.go b/internal/cmd/search_test.go index 90324bc..54a5870 100644 --- a/internal/cmd/search_test.go +++ b/internal/cmd/search_test.go @@ -5,10 +5,15 @@ import ( "path/filepath" "strings" "testing" + "time" "github.com/thevibeworks/ccx/internal/parser" ) +// sub is the default substring matcher; word is -w. +func sub(q string) textMatcher { return newTextMatcher(q, false) } +func word(q string) textMatcher { return newTextMatcher(q, true) } + func TestCountMatchingLines(t *testing.T) { dir := t.TempDir() path := filepath.Join(dir, "s.jsonl") @@ -19,13 +24,13 @@ func TestCountMatchingLines(t *testing.T) { if err := os.WriteFile(path, []byte(content), 0o644); err != nil { t.Fatal(err) } - if got := countMatchingLines(path, "pi-agent"); got != 2 { + if got, _ := countMatchingLines(path, sub("pi-agent")); got != 2 { t.Fatalf("case-insensitive matches: got %d, want 2", got) } - if got := countMatchingLines(path, "absent-term"); got != 0 { + if got, _ := countMatchingLines(path, sub("absent-term")); got != 0 { t.Fatalf("no-match count: got %d, want 0", got) } - if got := countMatchingLines(filepath.Join(dir, "missing.jsonl"), "x"); got != 0 { + if got, _ := countMatchingLines(filepath.Join(dir, "missing.jsonl"), sub("x")); got != 0 { t.Fatalf("missing file must count 0, got %d", got) } } @@ -43,10 +48,10 @@ func TestCountContentMatchesIncludesSubagents(t *testing.T) { if err := os.MkdirAll(subDir, 0o755); err != nil { t.Fatal(err) } - sub := `{"text":"goose in sidechain"} + sideLines := `{"text":"goose in sidechain"} {"text":"more goose here"} ` - if err := os.WriteFile(filepath.Join(subDir, "agent-1.jsonl"), []byte(sub), 0o644); err != nil { + if err := os.WriteFile(filepath.Join(subDir, "agent-1.jsonl"), []byte(sideLines), 0o644); err != nil { t.Fatal(err) } // Non-jsonl files (meta.json) and jsonl without the agent- prefix @@ -58,12 +63,18 @@ func TestCountContentMatchesIncludesSubagents(t *testing.T) { t.Fatal(err) } - if got := countContentMatches(main, "goose"); got != 3 { + if got, _, _ := countContentMatches(main, sub("goose"), false); got != 3 { t.Fatalf("main+subagent matches: got %d, want 3", got) } - if got := countContentMatches("", "goose"); got != 0 { + if got, _, _ := countContentMatches("", sub("goose"), false); got != 0 { t.Fatalf("empty path must count 0, got %d", got) } + if !sessionHasRawMatch(main, sub("sidechain")) { + t.Fatal("prefilter must see subagent files") + } + if sessionHasRawMatch(main, sub("absent")) { + t.Fatal("prefilter false positive") + } } type stubSessionParser struct{} @@ -100,13 +111,16 @@ func TestScanConversationTextSignalOnly(t *testing.T) { // 6 raw lines match; only 5 occurrences live in conversation text // (1 user + 2 assistant + 1 thinking + 1 sidechain). - if raw := countContentMatches(main, "deadman"); raw != 6 { - t.Fatalf("raw line matches: got %d, want 6", raw) + if raw, first, _ := countContentMatches(main, sub("deadman"), false); raw != 6 || !first.Equal(time.Date(2026, 8, 3, 0, 0, 0, 0, time.UTC)) { + t.Fatalf("raw line matches: got %d @ %v, want 6 @ 2026-08-03T00:00:00Z", raw, first) } - n, previews := scanConversationText(stubSessionParser{}, main, "deadman") + n, first, previews, _ := scanConversationText(stubSessionParser{}, main, sub("deadman"), false) if n != 5 { t.Fatalf("signal matches: got %d, want 5 (noise counted?)", n) } + if !first.Equal(time.Date(2026, 8, 3, 0, 0, 0, 0, time.UTC)) { + t.Fatalf("first hit: got %v, want the user prompt time", first) + } if len(previews) != maxContentPreviews { t.Fatalf("previews: got %d, want %d", len(previews), maxContentPreviews) } @@ -117,7 +131,7 @@ func TestScanConversationTextSignalOnly(t *testing.T) { t.Fatalf("second preview role: got %q, want assistant", previews[1].Role) } - if n, _ := scanConversationText(stubSessionParser{}, main, "auto-handoff"); n != 1 { + if n, _, _, _ := scanConversationText(stubSessionParser{}, main, sub("auto-handoff"), false); n != 1 { t.Fatalf("auto-handoff signal matches: got %d, want 1 (hook noise counted?)", n) } } @@ -138,7 +152,7 @@ func TestScanConversationTextSidechainRole(t *testing.T) { t.Fatal(err) } - n, previews := scanConversationText(stubSessionParser{}, main, "goose") + n, _, previews, _ := scanConversationText(stubSessionParser{}, main, sub("goose"), false) if n != 1 || len(previews) != 1 { t.Fatalf("sidechain match: got n=%d previews=%d, want 1/1", n, len(previews)) } @@ -189,7 +203,284 @@ func TestCountMatchingLinesOversizedLine(t *testing.T) { if err := os.WriteFile(path, []byte(content), 0o644); err != nil { t.Fatal(err) } - if got := countMatchingLines(path, "needle"); got != 1 { + if got, _ := countMatchingLines(path, sub("needle")); got != 1 { t.Fatalf("match after oversized line: got %d, want 1", got) } } + +// -w must stop "semantica" from matching "semantically" — the case that +// made 46 of 47 results noise — while still matching the term inside +// punctuation, hyphenated compounds, and CJK runs (no ASCII word +// boundary exists there, so none is demanded). +func TestTextMatcherWord(t *testing.T) { + cases := []struct { + q, text string + sub, w int + }{ + {"semantica", "semantically correct", 1, 0}, + {"semantica", "we chose Semantica.", 1, 1}, + {"semantica", "semantica-agi ships semantica", 2, 2}, + {"semantica", "(semantica)", 1, 1}, + {"pi-agent", "Pi-Agent uses ACP; pi-agents too", 2, 1}, + {"中文", "关于中文的讨论", 1, 1}, + {"fix bug", "fix bugs later; fix bug now", 2, 1}, + } + for _, c := range cases { + if got := sub(c.q).count(c.text); got != c.sub { + t.Errorf("sub(%q).count(%q) = %d, want %d", c.q, c.text, got, c.sub) + } + if got := word(c.q).count(c.text); got != c.w { + t.Errorf("word(%q).count(%q) = %d, want %d", c.q, c.text, got, c.w) + } + if (word(c.q).matches(c.text)) != (c.w > 0) { + t.Errorf("word(%q).matches(%q) disagrees with count %d", c.q, c.text, c.w) + } + } + // index feeds the preview window: it must point at the real match, + // not the substring inside a longer word. + if i, n := word("semantica").index("semantically, then semantica-agi"); i != 19 || n != len("semantica") { + t.Fatalf("word index: got %d/%d, want 19/9", i, n) + } + // literal() is the prefilter: a superset of word matches. + if !word("semantica").literal().matches("semantically") { + t.Fatal("literal prefilter must keep substring semantics") + } +} + +// indexFold/countFold replace ToLower-per-line on the raw scan; they +// must agree with the allocating form on ASCII and non-ASCII input. +func TestIndexFold(t *testing.T) { + cases := []struct { + s, q string + idx int + n int + }{ + {"tell me about Pi-Agent", "pi-agent", 14, 1}, + {"PI-AGENT pi-agent Pi-Agent", "pi-agent", 0, 3}, + {"nothing here", "absent", -1, 0}, + {"aaa", "aa", 0, 1}, + {"路径 and 路径", "路径", 0, 2}, + {"x", "", 0, 0}, + } + for _, c := range cases { + if got := indexFold(c.s, c.q); got != c.idx { + t.Errorf("indexFold(%q,%q) = %d, want %d", c.s, c.q, got, c.idx) + } + if got := countFold(c.s, c.q); got != c.n { + t.Errorf("countFold(%q,%q) = %d, want %d", c.s, c.q, got, c.n) + } + if c.q != "" { + if want := strings.Index(strings.ToLower(c.s), c.q); want != c.idx { + t.Errorf("test expectation drift for %q: ToLower form gives %d", c.q, want) + } + } + } + // A window that cuts a multi-byte rune must not match. + if indexFold("cafés", "\xc3s") != -1 { + t.Fatal("partial rune must not fold-match") + } +} + +func TestLineTimestamp(t *testing.T) { + // The tool result embeds another object's timestamp before the + // top-level one; only the top-level field counts. + line := `{"type":"user","message":{"content":[{"type":"tool_result","content":"{\"timestamp\":\"2020-01-01T00:00:00Z\"}"}]},"timestamp":"2026-08-03T01:02:03.500Z"}` + if got := lineTimestamp(line); !got.Equal(time.Date(2026, 8, 3, 1, 2, 3, 500000000, time.UTC)) { + t.Fatalf("lineTimestamp: got %v", got) + } + if !lineTimestamp(`{"no":"ts"}`).IsZero() || !lineTimestamp(`not json`).IsZero() { + t.Fatal("missing/invalid timestamp must be zero") + } +} + +// A session whose summary matches must still carry its content +// evidence: previously the summary hit short-circuited the scan, so +// the one session where the term was actually discussed was the one +// result with no matches, previews, or first-hit time. +func TestSessionSearcherSummaryHitKeepsContent(t *testing.T) { + dir := t.TempDir() + main := filepath.Join(dir, "abc-123.jsonl") + lines := strings.Join([]string{ + `{"type":"user","uuid":"u1","timestamp":"2026-08-18T07:00:00Z","message":{"role":"user","content":[{"type":"text","text":"when did we mention semantica?"}]}}`, + `{"type":"assistant","uuid":"a1","parentUuid":"u1","timestamp":"2026-08-18T07:00:05Z","message":{"role":"assistant","content":[{"type":"text","text":"semantically, never; semantica itself: now"}]}}`, + }, "\n") + "\n" + if err := os.WriteFile(main, []byte(lines), 0o644); err != nil { + t.Fatal(err) + } + end := time.Date(2026, 8, 18, 8, 0, 0, 0, time.UTC) + sess := &parser.Session{ID: "abc-123", FilePath: main, Summary: "trace semantica mentions", EndTime: end} + + // Name-only search: summary hit, no content fields. + r := sessionSearcher{m: word("semantica")}.match(sess, "proj") + if r == nil || r.Type != "session" || r.Matches != 0 || r.FirstHit != "" { + t.Fatalf("summary-only match: got %+v", r) + } + + // Content on: same row, now with evidence. Word mode counts 2 + // (not the "semantically" in the reply). + r = sessionSearcher{m: word("semantica"), content: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || r.Type != "session" || r.Priority != 2 { + t.Fatalf("summary+content match must stay a session hit, got %+v", r) + } + if r.Matches != 2 || len(r.Previews) != 2 || r.Previews[0].Role != "user" { + t.Fatalf("content evidence missing on summary hit: %+v", r) + } + if r.FirstHit != "2026-08-18T07:00:00Z" || !r.lastTime.Equal(end) { + t.Fatalf("first/last: got %q / %v", r.FirstHit, r.lastTime) + } + + // Substring mode counts the noise too; -w is what makes the + // count trustworthy. + r = sessionSearcher{m: sub("semantica"), content: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || r.Matches != 3 { + t.Fatalf("substring count: got %+v", r) + } + + // No summary hit, no content hit: nil, not an empty row. + if r := (sessionSearcher{m: word("absent"), content: true, parser: stubSessionParser{}}).match(sess, "proj"); r != nil { + t.Fatalf("miss must be nil, got %+v", r) + } + + // --raw: line count plus first-hit from the raw timestamp. + r = sessionSearcher{m: sub("semantica"), content: true, raw: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || r.Matches != 2 || r.FirstHit != "2026-08-18T07:00:00Z" { + t.Fatalf("raw match: got %+v", r) + } +} + +// matchAll must return hits in candidate order regardless of which +// worker finishes first — stable ranking depends on it. +func TestSessionSearcherMatchAllOrder(t *testing.T) { + var cands []sessionCandidate + for i := 0; i < 50; i++ { + summary := "nothing" + if i%3 == 0 { + summary = "goose" + } + cands = append(cands, sessionCandidate{ + session: &parser.Session{ID: "id-" + strings.Repeat("x", i%7) + string(rune('a'+i%26)), Summary: summary}, + project: "p", + }) + } + hits := sessionSearcher{m: sub("goose")}.matchAll(cands, 4) + if len(hits) != 17 { + t.Fatalf("hits: got %d, want 17", len(hits)) + } + for i := 1; i < len(hits); i++ { + if hits[i-1].Session > hits[i].Session && cands[0].session != nil { + // IDs are not monotonic by construction; check discovery + // order via summary sequence instead. + break + } + } + want := 0 + for _, c := range cands { + if c.session.Summary != "goose" { + continue + } + if hits[want].Session != truncateID(c.session.ID, 8) { + t.Fatalf("hit %d out of order: got %s want %s", want, hits[want].Session, c.session.ID) + } + want++ + } +} + +func TestSortSearchResults(t *testing.T) { + ts := func(d int) time.Time { return time.Date(2026, 8, d, 0, 0, 0, 0, time.UTC) } + mk := func(name string, prio, hits, first, last int) searchResult { + r := searchResult{Session: name, Priority: prio, Matches: hits} + if first > 0 { + r.firstHit = ts(first) + } + if last > 0 { + r.lastTime = ts(last) + } + return r + } + order := func(rs []searchResult) string { + var ids []string + for _, r := range rs { + ids = append(ids, r.Session) + } + return strings.Join(ids, " ") + } + base := func() []searchResult { + return []searchResult{ + mk("proj", 1, 0, 0, 0), + mk("late-many", 3, 9, 15, 17), + mk("early-few", 3, 1, 3, 10), + mk("summary", 2, 0, 0, 12), + } + } + + rs := base() + sortSearchResults(rs, "hits") + if got := order(rs); got != "proj summary late-many early-few" { + t.Fatalf("hits order: %s", got) + } + rs = base() + sortSearchResults(rs, "first") + if got := order(rs); got != "early-few late-many proj summary" { + t.Fatalf("first order: %s (timeless must trail in hits order)", got) + } + rs = base() + sortSearchResults(rs, "last") + if got := order(rs); got != "late-many summary early-few proj" { + t.Fatalf("last order: %s", got) + } +} + +// --hits turns a search into citations: one row per matching message +// with the anchors (message id, time, role) needed to walk back to +// it, oldest first across sessions. Under --raw the unit is a line. +func TestSessionSearcherHits(t *testing.T) { + dir := t.TempDir() + main := filepath.Join(dir, "abc-123.jsonl") + lines := strings.Join([]string{ + `{"type":"user","uuid":"u1","timestamp":"2026-08-18T07:00:00Z","message":{"role":"user","content":[{"type":"text","text":"when did we mention semantica?"}]}}`, + `{"type":"assistant","uuid":"a1","parentUuid":"u1","timestamp":"2026-08-18T07:00:05Z","message":{"role":"assistant","content":[{"type":"text","text":"semantically, never"},{"type":"text","text":"semantica itself: now, and semantica again"}]}}`, + `{"type":"user","uuid":"t1","parentUuid":"a1","timestamp":"2026-08-18T07:00:06Z","message":{"role":"user","content":[{"type":"tool_result","content":"semantica in a tool result"}]}}`, + }, "\n") + "\n" + if err := os.WriteFile(main, []byte(lines), 0o644); err != nil { + t.Fatal(err) + } + sess := &parser.Session{ID: "abc-123", FilePath: main, Summary: "s"} + + r := sessionSearcher{m: word("semantica"), content: true, hits: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || r.Matches != 3 || len(r.hits) != 2 { + t.Fatalf("hits: got %+v", r) + } + h := r.hits[0] + if h.MessageID != "u1" || h.Role != "user" || h.Matches != 1 || h.Session != "abc-123" || h.Path != main || h.Project != "proj" { + t.Fatalf("first hit anchors: %+v", h) + } + if !h.Time.Equal(time.Date(2026, 8, 18, 7, 0, 0, 0, time.UTC)) || !strings.Contains(h.Quote, "semantica") { + t.Fatalf("first hit time/quote: %+v", h) + } + if r.hits[1].MessageID != "a1" || r.hits[1].Matches != 2 || !strings.Contains(r.hits[1].Quote, "semantica itself") { + t.Fatalf("second hit must count both blocks and quote the first real match, got %+v", r.hits[1]) + } + + // Without --hits nothing extra is collected. + if r := (sessionSearcher{m: word("semantica"), content: true, parser: stubSessionParser{}}).match(sess, "proj"); len(r.hits) != 0 { + t.Fatalf("hits collected without --hits: %d", len(r.hits)) + } + + // --raw --hits: every matching line, including the tool result, + // anchored by the line's own uuid/type/timestamp. + r = sessionSearcher{m: sub("semantica"), content: true, raw: true, hits: true, parser: stubSessionParser{}}.match(sess, "proj") + if r == nil || len(r.hits) != 3 || r.hits[2].MessageID != "t1" || r.hits[2].Role != "user" { + t.Fatalf("raw hits: got %+v", r) + } + + // collectHits orders across sessions by time, zero times last. + later := searchResult{hits: []searchHit{{MessageID: "z", Time: time.Date(2026, 8, 19, 0, 0, 0, 0, time.UTC)}, {MessageID: "none"}}} + all := collectHits([]searchResult{later, *r}) + if len(all) != 5 || all[0].MessageID != "u1" || all[3].MessageID != "z" || all[4].MessageID != "none" { + ids := []string{} + for _, h := range all { + ids = append(ids, h.MessageID) + } + t.Fatalf("collectHits order: %v", ids) + } +} diff --git a/internal/cmd/sessions.go b/internal/cmd/sessions.go index 5899f20..fb11da7 100644 --- a/internal/cmd/sessions.go +++ b/internal/cmd/sessions.go @@ -50,7 +50,7 @@ var ( func init() { sessionsCmd.Flags().StringVar(&sessionsSort, "sort", "time", "sort by: time, messages, prompts, tokens") - sessionsCmd.Flags().IntVar(&sessionsLimit, "limit", 20, "limit number of sessions (0 = no limit)") + sessionsCmd.Flags().IntVarP(&sessionsLimit, "limit", "n", 20, "limit number of sessions (0 = no limit)") sessionsCmd.Flags().BoolVar(&sessionsJSON, "json", false, "output as JSON") sessionsCmd.Flags().StringVarP(&sessionsProvider, "provider", "p", "", "filter by provider: cc, cx, gx, all") sessionsCmd.Flags().StringVarP(&sessionsSearch, "search", "s", "", "search in session summaries") diff --git a/skills/ccx/SKILL.md b/skills/ccx/SKILL.md index 7495ecb..0cb35f3 100644 --- a/skills/ccx/SKILL.md +++ b/skills/ccx/SKILL.md @@ -152,8 +152,17 @@ Features: ccx search "error handling" # All providers ccx search --provider=cc "auth bug" # Claude Code only ccx search --after=2026-03-01 "deploy" # Date filtered +ccx search --content "deploy" # Also scan conversation text (user + assistant) +ccx search --content -w --sort first X # Whole word only, earliest mention first (FIRST column) +ccx search --content -w --hits X # Every mention as a citation: time, session, role, message id, quote +ccx search --raw "deploy" # Grep parity over raw transcript lines ``` +`-w` matters when the term prefixes a common word ("semantica" vs +"semantically"); `--json` carries `matches`, `previews`, `first_hit`. +Cite from `--hits --json` (`message_id`, `time`, `quote`) rather than +from a session-level count when a claim needs evidence. + Web search supports provider prefixes: `cc: auth bug`, `cx: codex query`, `gx: grok query` ## Fork Session From 88197747e3a5b63af39c106fc8f6b97edb533f31 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 05:42:03 -0700 Subject: [PATCH 03/13] =?UTF-8?q?feat(related):=20session=20connections=20?= =?UTF-8?q?=E2=80=94=20fork,=20mentions,=20handoff,=20shared=20files?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sessions were islands. The joins between them — the handoff a later session picked up, the fork that carried a conversation into a new file, the second agent on the same files at the same time, the session that said "see 736a7bac" — are all in the transcripts, but nothing computed them (docs/design/0006-session-connections.md). - New `ccx related [session]`: the anchor's connections to the other sessions of its workspace, deterministic and evidence-backed: forked_from/fork_of (shared message ids), mentions/mentioned_by (id prefix in conversation text, quoted), handoff_from/handoff_to (baton file written by one, read by the other later), builds_on/built_on_by (file edited by one, then touched by the other), overlaps, previous/next. Strength is a band; path lists are capped with count kept and `truncated` set; --json is ccx.related.v1 with message id, time, path, quote per relation. - `ccx trace --full` carries the same list as `related`. - Profiles are built once per workspace session on the search worker pool with TTY progress; the parse cache makes repeats ~1s. - Fix: heredoc bodies were scanned as shell redirects, so a Go `if n > 0` or a markdown `> 2026-08-18` inside `python3 - <<'EOF'` became "edited files" in trace files_edited (and would have been junk builds_on evidence). Bodies are stripped before the redirect scan; regression test names the symptom. Tests: related_test.go (handoff/builds_on direction and evidence pairing, fork + mentions incl. tool-result exclusion, overlap window, ordering, self-skip, path cap, baton paths, id prefix); TestExtractRedirectPathsIgnoresHeredocBodies. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 2 + README.md | 1 + docs/design/0006-session-connections.md | 74 ++++ docs/schema.md | 3 +- internal/cmd/related.go | 294 ++++++++++++++ internal/cmd/root.go | 1 + internal/cmd/trace.go | 13 + internal/trace/analysis.go | 34 ++ internal/trace/analysis_test.go | 22 ++ internal/trace/related.go | 485 ++++++++++++++++++++++++ internal/trace/related_test.go | 227 +++++++++++ internal/trace/types.go | 9 +- skills/ccx/SKILL.md | 18 +- 13 files changed, 1179 insertions(+), 4 deletions(-) create mode 100644 docs/design/0006-session-connections.md create mode 100644 internal/cmd/related.go create mode 100644 internal/trace/related.go create mode 100644 internal/trace/related_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 799eb99..e3afa6c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ## [Unreleased] ### Added +- **`ccx related [session]`: which sessions connect to this one, and how.** Sessions were islands; the joins were in the transcripts but nothing computed them. `related` derives, deterministically and with evidence, the anchor session's connections to the other sessions of its workspace: `forked_from`/`fork_of` (the transcripts share message ids — Claude Code fork and `ccx fork` copy history verbatim), `mentions`/`mentioned_by` (a session id named in conversation text, quoted), `handoff_from`/`handoff_to` (a baton file — HANDOFF.md, handoffs/, devlog, PLAN.md — written by one and read by the other later), `builds_on`/`built_on_by` (a workspace file edited by one, then read or edited by the other), `overlaps` (concurrent), `previous`/`next`. Strength is a band (strong/medium/weak), never a score; path lists are capped with the count kept and `truncated` set; `--json` (`ccx.related.v1`) carries message id, time, path, and quote per relation. `ccx trace --full` gains the same list as `related`. Design: docs/design/0006-session-connections.md. - **`ccx search -w/--word` matches whole words.** Matching was substring-only, so a term that prefixes a common word was unanswerable: `search --content semantica` returned 47 sessions, 46 of them "semantic*ally*", and ccx alone could not tell 0 real hits from 46 (docs/devlog/2026-08-18-search-word-boundary-dogfood.org). `-w` demands an ASCII word boundary on each side of the query that starts/ends with a word character (so "semantica-agi" and "(semantica)" still hit; CJK queries are unaffected) and applies to names, summaries, conversation text, and `--raw` lines alike. - **`ccx search --hits` turns matches into citations.** One row per matching message — time, session, role, message id, quote — oldest first across sessions, `-n`-capped with a visible "showing N of M". The anchors are the same ones `trace` and `view` use, so a claim built on a search can point at its evidence (design: docs/design/0005-evidence-citations-lessons-from-semantica.md). Under `--raw` the unit is a transcript line, anchored by its own `uuid`/`type`/`timestamp`. - **`ccx search --content` reports when: `FIRST` column, `first_hit` in `--json`, `--sort first|last|hits`.** "When did we first mention X" needs the earliest matching message and an oldest-first order; results only carried session end time and ranked by hit count. Each content hit now records the timestamp of its earliest matching message (parsed messages by default; the raw line's top-level `timestamp` under `--raw`), printed as `FIRST` and sortable with `--sort first`; `--sort last` orders by session activity; `--sort hits` is the old order and the default. @@ -16,6 +17,7 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) - **`ccx search --content` is ~10x faster and shows progress.** The scan ran on one core and lowercased every transcript line: 6m18s cold / 1m14s warm over a 3.5 GB store, silent throughout. Sessions now scan on a bounded worker pool (up to 8), the raw prefilter matches case-insensitively without allocating and stops at the first hit, and a `scanning N/M sessions` line ticks on stderr when it is a terminal. Same store, warm: 7.6s. ### Fixed +- **Heredoc bodies no longer count as shell redirects.** `extractRedirectPaths` scanned the whole Bash command, so a Go `if n > 0` or a markdown `> 2026-08-18` inside `python3 - <<'EOF'` / `cat > f <<'EOF'` became "edited files" (`.../0`, `.../2026-08-18`) in `trace` `files_edited` and in session connections. Heredoc bodies are stripped before the redirect scan. - **`-n` is the `--limit` shorthand everywhere.** Only `search` had it; `sessions -n 2` failed with "unknown shorthand flag". `sessions`, `projects`, and `log` now accept `-n` too. - **A session whose summary matched dropped its content evidence.** The summary hit short-circuited the content scan, so under `--content` the session where the term was actually discussed could be the one result with no hit count, previews, or first-hit time. Summary hits stay typed `session` but now carry the content fields. - **Codex 0.147 conversations render again.** Codex moved its UI-facing user and assistant records from legacy `event_msg.user_message` / `agent_message` events to canonical `event_msg.item_completed` TurnItems. The Codex adapter now selects one conversation source per rollout, reads stable `UserMessage` / `AgentMessage` item IDs and content, keeps legacy rollouts working, and never mistakes raw `response_item` model input (which can contain injected instruction envelopes) for a human prompt. Discovery metadata, terminal view, web, export, search, and trace now agree; the parse-cache format is bumped so upgrades cannot serve blank cached sessions. diff --git a/README.md b/README.md index f83d1b7..56d4cac 100644 --- a/README.md +++ b/README.md @@ -78,6 +78,7 @@ ccx view [session] # View in terminal ccx export --shape brief # Export conversation-only HTML ccx export --shape human # Only the human's turns, citable ccx trace [session] -o trace.json # Extract evidence for context folding +ccx related [session] # Which sessions connect to this one (fork, handoff, mentions, shared files) ccx log --scope yesterday --tz +8 --all --json # Time-sliced log evidence ccx search "auth bug" # Search across sessions + memory ccx search --content -w --sort first goose # Whole-word content hits, earliest first diff --git a/docs/design/0006-session-connections.md b/docs/design/0006-session-connections.md new file mode 100644 index 0000000..d82b869 --- /dev/null +++ b/docs/design/0006-session-connections.md @@ -0,0 +1,74 @@ +# Design: Session connections + +**Created**: 2026-08-18 +**Status**: Accepted +**CLI**: `ccx related [session]`; `related` in `ccx trace --full` +**Follows**: docs/design/0005-evidence-citations-lessons-from-semantica.md +(principle 2: every claim carries a quote + location; principle 4: +supersedes vs derives-from) + +## Problem + +ccx treats sessions as islands. The evidence for what happened across a +piece of work — the handoff a later session picked up, the fork that +carried a conversation into a new session, the second agent working the +same files at the same time, the session that said "see 736a7bac" — is +all in the transcripts, but nothing joins it. Every "how did we get +here" question that spans sessions falls back to eyeballing timestamps. + +The MineContext framing (capture → process → consume) puts this in the +process layer: capture is free (the JSONL is already there); the missing +processing is *connect*. Semantica's framing: a decision chain that +cannot cross a session boundary is truncated at exactly the point where +context was lost. + +## What a connection is + +A connection is a deterministic relation between two sessions, backed +by evidence a reader can walk to (message id, time, path, quote). No +LLM, no similarity scores. Relations are stated from the anchor +session's point of view: + +| Relation | Signal | Evidence carried | Strength | +|---|---|---|---| +| `forked_from` / `fork_of` | the two transcripts share message UUIDs (Claude Code fork and `ccx fork` copy history verbatim); the earlier session is the origin | shared count, first shared uuid | strong | +| `mentions` / `mentioned_by` | conversation text contains ≥8 hex chars matching the other session's id prefix | message id, time, quote | strong | +| `handoff_from` / `handoff_to` | a baton file (`HANDOFF*.md`, `*/handoffs/*`, `*devlog*`, `PLAN.md`, `TODO.md`) written by one session and read by the other later | path, writer msg id + time, reader msg id + time | strong | +| `builds_on` / `built_on_by` | a workspace file edited by one session and read or edited by the other later | up to 5 paths with both anchors, total count | medium | +| `overlaps` | time windows intersect (concurrent agents) | overlap window | medium | +| `previous` / `next` | nearest earlier / later session in the same project | start time | weak | + +Strength is a band, never a decimal (0005 principle 7). A pair can carry +several relations; the pair's strength is the strongest one. + +Scope: sessions in the same project (workspace) across providers. +`--all` widens id resolution like `trace --all`; relation search stays +per project because file-based signals only mean something inside one +workspace. + +## Surface + +``` +ccx related # latest workspace session +ccx related 736a7bac # by id / prefix / @N +ccx related --json # full evidence +ccx trace --full # bundle gains "related" +``` + +Text output: one row per related session, strongest first, then by +time: STRENGTH, SESSION, RELATIONS, WHEN, EVIDENCE (one bounded line). +Silent-cap rule holds: `-n` limits rows with "showing N of M". + +## Cost + +Every session in the project is parsed once (parse cache makes repeats +cheap); scan runs on the same bounded worker pool as `search`. Progress +on stderr when it is a terminal. + +## Non-goals + +- Semantic similarity between sessions (skills' job). +- Cross-workspace file links (paths are only comparable inside one + workspace). +- Persisting a graph. The relations are recomputed from the transcripts; + the transcripts are the store. diff --git a/docs/schema.md b/docs/schema.md index 1703fc3..b3b30aa 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -14,7 +14,8 @@ they do not know. |------------------|------------------------------|----------| | `ccx.outline.v1` | `ccx trace --json` | Session skeleton: every turn and step headline with rollups. Read this first; it always fits. | | `ccx.turn.v1` | `ccx trace --turn N` | One turn with full step evidence, plus the sidechain entries that turn references, plus warnings. | -| `ccx.trace.v2` | `ccx trace --full` | Complete evidence bundle: all turns/steps, sidechains, git correlation, workspace context, stats, warnings. Large. | +| `ccx.trace.v2` | `ccx trace --full` | Complete evidence bundle: all turns/steps, sidechains, git correlation, workspace context, `related` sessions, stats, warnings. Large. | +| `ccx.related.v1` | `ccx related --json` | The anchor session's connections to the other sessions of its workspace: `related[]` of `{session_id, provider, summary, start, end, strength, relations[]}`, plus `total`/`shown`. Each relation is `{kind, count?, paths?, evidence[], truncated?}`; evidence items are `{session_id, message_id, time, path?, quote?}`. Kinds: `forked_from`/`fork_of`, `mentions`/`mentioned_by`, `handoff_from`/`handoff_to`, `builds_on`/`built_on_by`, `overlaps`, `previous`/`next`. Strength is `strong`/`medium`/`weak`. | | `ccx.log.v1` | `ccx log --json`, `ccx insight --json` | Time-scoped records across sessions with pre-computed `days[]` / `providers[]` / `workspaces[]` aggregates. | ## Versioning policy diff --git a/internal/cmd/related.go b/internal/cmd/related.go new file mode 100644 index 0000000..6125591 --- /dev/null +++ b/internal/cmd/related.go @@ -0,0 +1,294 @@ +package cmd + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "sync" + "text/tabwriter" + "time" + + "github.com/spf13/cobra" + + "github.com/thevibeworks/ccx/internal/catalog" + "github.com/thevibeworks/ccx/internal/parser" + "github.com/thevibeworks/ccx/internal/provider" + "github.com/thevibeworks/ccx/internal/trace" +) + +var relatedCmd = &cobra.Command{ + Use: "related [session]", + Short: "Which sessions connect to this one, and how", + Long: `Which sessions connect to this one, and how. + +Sessions are islands in the log; the connections between them are +in the transcripts but nothing joins them. This command joins them, +deterministically, from the anchor session's point of view: + + forked_from / fork_of the transcripts share message ids + mentions / mentioned_by text names the other session's id + handoff_from / handoff_to a baton file (HANDOFF.md, handoffs/, + devlog, PLAN.md) written by one, read + by the other later + builds_on / built_on_by a workspace file edited by one, then + read or edited by the other + overlaps the two ran at the same time + previous / next nearest neighbours in the workspace + +Every relation carries evidence (message id, time, path, quote) in +--json. Strength is a band — strong, medium, weak — never a score. +Scope is the anchor session's workspace, all providers. + +Examples: + ccx related Connections of the latest workspace session + ccx related 736a7bac A specific session + ccx related --json Full evidence for scripts and skills`, + Args: cobra.MaximumNArgs(1), + RunE: runRelated, +} + +var ( + relatedProject string + relatedAll bool + relatedJSON bool + relatedLimit int +) + +func init() { + relatedCmd.Flags().StringVarP(&relatedProject, "project", "p", "", "project name") + relatedCmd.Flags().BoolVar(&relatedAll, "all", false, "resolve the session across all projects") + relatedCmd.Flags().BoolVar(&relatedJSON, "json", false, "output as JSON") + relatedCmd.Flags().IntVarP(&relatedLimit, "limit", "n", 20, "max related sessions (0 = no limit)") + rootCmd.AddCommand(relatedCmd) +} + +func runRelated(cmd *cobra.Command, args []string) error { + backend := provider.Default() + + var session *parser.Session + var err error + if len(args) == 0 { + session, err = latestTraceSession(backend, relatedAll) + if err != nil { + return fmt.Errorf("session: %w", err) + } + } else { + projectName, sessionID := parseSessionArg(args[0]) + if relatedProject != "" { + projectName = relatedProject + } + query, err := sessionLookupQuery(projectName, relatedAll) + if err != nil { + return err + } + session, err = resolveSessionInQuery(backend, query, sessionID) + if err != nil { + return err + } + } + if session == nil { + return fmt.Errorf("no session found") + } + + related, warnings, err := relateSession(backend, session) + if err != nil { + return err + } + for _, w := range warnings { + fmt.Fprintf(os.Stderr, "warning: %s\n", w.Message) + } + + total := len(related) + if relatedLimit > 0 && total > relatedLimit { + fmt.Fprintf(os.Stderr, "showing %d of %d related sessions (raise with -n)\n", relatedLimit, total) + related = related[:relatedLimit] + } + + if relatedJSON { + enc := json.NewEncoder(os.Stdout) + enc.SetIndent("", " ") + return enc.Encode(map[string]any{ + "kind": "ccx.related.v1", + "session": map[string]any{"id": session.ID, "provider": session.Provider, "project": session.ProjectName, "path": session.FilePath}, + "related": related, + "total": total, + "shown": len(related), + }) + } + return printRelated(session, related, total) +} + +// relateSession profiles every session of the anchor's workspace and +// returns the anchor's connections. Shared by `ccx related` and the +// trace --full bundle. +func relateSession(backend provider.Backend, anchor *parser.Session) ([]trace.RelatedSession, []trace.TraceWarning, error) { + query := catalog.SessionQuery{Scope: catalog.ScopeProject, ProjectName: anchor.ProjectName} + if strings.TrimSpace(anchor.CWD) != "" { + query = catalog.SessionQuery{Scope: catalog.ScopeWorkspace, WorkspacePath: anchor.CWD} + } + sessions, err := backend.ListSessions(query.WithoutLimit().WithoutProviderFilter()) + if err != nil { + return nil, nil, fmt.Errorf("list workspace sessions: %w", err) + } + // The anchor may be missing from the workspace listing when its + // cwd differs from the project path (a fork into another dir); + // it always takes part. + found := false + for _, s := range sessions { + if s.FilePath == anchor.FilePath { + found = true + break + } + } + if !found { + sessions = append(sessions, anchor) + } + + profiles := make([]*trace.SessionProfile, len(sessions)) + var warnings []trace.TraceWarning + var warnMu sync.Mutex + progress := newScanProgress(len(sessions), true) + var wg sync.WaitGroup + next := make(chan int) + for w := 0; w < searchWorkers(true); w++ { + wg.Add(1) + go func() { + defer wg.Done() + for i := range next { + full, err := backend.ParseSession(sessions[i].FilePath) + if err != nil { + warnMu.Lock() + warnings = append(warnings, trace.TraceWarning{Kind: "related_parse_failed", Message: fmt.Sprintf("skipping %s: %v", filepath.Base(sessions[i].FilePath), err)}) + warnMu.Unlock() + } else { + profiles[i] = trace.ProfileSession(full) + } + progress.tick() + } + }() + } + for i := range sessions { + next <- i + } + close(next) + wg.Wait() + progress.done() + + var anchorProfile *trace.SessionProfile + others := make([]*trace.SessionProfile, 0, len(profiles)) + for i, p := range profiles { + if p == nil { + continue + } + if sessions[i].FilePath == anchor.FilePath { + anchorProfile = p + continue + } + others = append(others, p) + } + if anchorProfile == nil { + return nil, warnings, fmt.Errorf("could not parse anchor session %s", anchor.ID) + } + return trace.RelateSessions(anchorProfile, others), warnings, nil +} + +func printRelated(anchor *parser.Session, related []trace.RelatedSession, total int) error { + // Mirror the trace header: workspace basename beats the encoded + // project dir name. + where := anchor.ProjectName + if strings.HasPrefix(where, "-") { + where = parser.GetProjectDisplayName(where) + } + if strings.TrimSpace(anchor.CWD) != "" { + where = filepath.Base(anchor.CWD) + } + fmt.Printf("related to %s (%s): %d session(s)\n", truncateID(anchor.ID, 8), cleanDisplayText(where), total) + if len(related) == 0 { + return nil + } + w := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0) + fmt.Fprintln(w, "STRENGTH\tSESSION\tPROVIDER\tRELATIONS\tSTART\tEVIDENCE") + for _, r := range related { + kinds := make([]string, 0, len(r.Relations)) + for _, rel := range r.Relations { + k := rel.Kind + if rel.Count > 1 && (rel.Kind == trace.RelBuildsOn || rel.Kind == trace.RelBuiltOnBy || rel.Kind == trace.RelHandoffFrom || rel.Kind == trace.RelHandoffTo) { + k = fmt.Sprintf("%s(%d)", k, rel.Count) + } + kinds = append(kinds, k) + } + fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", + r.Strength, truncateID(r.SessionID, 8), providerTag(r.Provider), strings.Join(kinds, ","), + formatRelatedTime(r.Start), truncateDisplay(cleanDisplayText(relationEvidenceLine(r)), 96)) + } + return w.Flush() +} + +// relationEvidenceLine renders the strongest relation's evidence as one +// bounded line; the rest is in --json. +func relationEvidenceLine(r trace.RelatedSession) string { + if len(r.Relations) == 0 { + return "" + } + rel := r.Relations[0] + for _, cand := range r.Relations[1:] { + if strengthRankOf(cand.Kind) < strengthRankOf(rel.Kind) { + rel = cand + } + } + switch rel.Kind { + case trace.RelForkedFrom, trace.RelForkOf: + id := "" + if len(rel.Evidence) > 0 { + id = truncateID(rel.Evidence[0].MessageID, 8) + } + return fmt.Sprintf("%d shared message ids (e.g. %s)", rel.Count, id) + case trace.RelMentions, trace.RelMentionedBy: + if len(rel.Evidence) > 0 { + return fmt.Sprintf("%s@%s: %s", truncateID(rel.Evidence[0].SessionID, 8), formatRelatedTime(rel.Evidence[0].Time), rel.Evidence[0].Quote) + } + case trace.RelHandoffFrom, trace.RelHandoffTo, trace.RelBuildsOn, trace.RelBuiltOnBy: + if len(rel.Evidence) >= 2 { + more := "" + if rel.Count > 1 { + more = fmt.Sprintf(" (+%d more)", rel.Count-1) + } + return fmt.Sprintf("%s%s: written %s@%s, touched %s@%s", + shortPath(rel.Evidence[0].Path), more, + truncateID(rel.Evidence[0].SessionID, 8), formatRelatedTime(rel.Evidence[0].Time), + truncateID(rel.Evidence[1].SessionID, 8), formatRelatedTime(rel.Evidence[1].Time)) + } + case trace.RelOverlaps: + if len(rel.Evidence) >= 2 { + return fmt.Sprintf("both active %s - %s", formatRelatedTime(rel.Evidence[0].Time), formatRelatedTime(rel.Evidence[1].Time)) + } + } + return "-" +} + +func strengthRankOf(kind string) int { + switch kind { + case trace.RelForkedFrom, trace.RelForkOf, trace.RelMentions, trace.RelMentionedBy, trace.RelHandoffFrom, trace.RelHandoffTo: + return 0 + case trace.RelBuildsOn, trace.RelBuiltOnBy, trace.RelOverlaps: + return 1 + } + return 2 +} + +func shortPath(path string) string { + parts := strings.Split(filepath.ToSlash(path), "/") + if len(parts) <= 3 { + return path + } + return ".../" + strings.Join(parts[len(parts)-3:], "/") +} + +func formatRelatedTime(t time.Time) string { + if t.IsZero() { + return "-" + } + return t.Local().Format("2006-01-02 15:04") +} diff --git a/internal/cmd/root.go b/internal/cmd/root.go index f5a0363..df02c7b 100644 --- a/internal/cmd/root.go +++ b/internal/cmd/root.go @@ -42,6 +42,7 @@ CLI commands: ccx view View session in terminal ccx export -f html Export to HTML/Markdown/Org ccx trace What the agent did: turn/step outline + drill-down + ccx related Which sessions connect to this one, and how ccx log Slice raw session logs by time scope ccx skills install Install bundled agent skills matching this binary diff --git a/internal/cmd/trace.go b/internal/cmd/trace.go index 11e0408..761c739 100644 --- a/internal/cmd/trace.go +++ b/internal/cmd/trace.go @@ -114,6 +114,19 @@ func runTrace(cmd *cobra.Command, args []string) error { }) } + // Session connections cost a parse of every workspace session, so + // they ride only in the full bundle (docs/design/0006). + if traceFull { + related, warnings, err := relateSession(backend, session) + result.Warnings = append(result.Warnings, warnings...) + if err != nil { + result.Warnings = append(result.Warnings, trace.TraceWarning{Kind: "related_failed", Message: err.Error()}) + fmt.Fprintf(os.Stderr, "warning: session connections failed: %v\n", err) + } else { + result.Related = related + } + } + output, err := renderTrace(result) if err != nil { return err diff --git a/internal/trace/analysis.go b/internal/trace/analysis.go index 17a6bef..3e793de 100644 --- a/internal/trace/analysis.go +++ b/internal/trace/analysis.go @@ -603,10 +603,44 @@ func extractPatchPaths(patch string) []string { var redirectPattern = regexp.MustCompile(`(?:>>?|\btee(?:\s+-a)?)\s+([^\s;|&<>]+)`) +// heredocStart matches a heredoc operator and captures its delimiter: +// < 0`, a blockquote +// `> 2026-08-18`) is not read as an output redirect. The body starts +// after the line carrying the operator and ends at the line equal to +// the delimiter (leading tabs allowed for <<-). +func stripHeredocs(command string) string { + if !strings.Contains(command, "<<") { + return command + } + lines := strings.Split(command, "\n") + var out []string + for i := 0; i < len(lines); i++ { + line := lines[i] + out = append(out, line) + m := heredocStart.FindStringSubmatch(line) + if m == nil { + continue + } + delim := m[2] + for i+1 < len(lines) { + i++ + if strings.TrimLeft(lines[i], "\t") == delim { + break + } + } + } + return strings.Join(out, "\n") +} + func extractRedirectPaths(command string) []string { if !strings.ContainsAny(command, ">") && !strings.Contains(command, "tee") { return nil } + command = stripHeredocs(command) matches := redirectPattern.FindAllStringSubmatch(command, -1) var paths []string for _, m := range matches { diff --git a/internal/trace/analysis_test.go b/internal/trace/analysis_test.go index f68e113..4200a1f 100644 --- a/internal/trace/analysis_test.go +++ b/internal/trace/analysis_test.go @@ -548,3 +548,25 @@ func TestCommandSummaryCollapsesAndBounds(t *testing.T) { t.Fatal("nil input must yield empty summary") } } + +// Heredoc bodies are data, not shell: a Go comparison or a markdown +// blockquote piped through `python3 - <<'EOF'` / `cat > f <<'EOF'` must +// not become "edited files" (they surfaced as ".../0" and +// ".../2026-08-18" in files_edited and in session connections). +func TestExtractRedirectPathsIgnoresHeredocBodies(t *testing.T) { + cmd := "cd repo && python3 - <<'EOF'\nif first > 0:\n pass\nx = a > b\nEOF\ncat > out.md < 2026-08-18 quote\nEOF\necho done > final.log" + got := extractRedirectPaths(cmd) + want := []string{"out.md", "final.log"} + if strings.Join(got, ",") != strings.Join(want, ",") { + t.Fatalf("redirect paths: got %v, want %v", got, want) + } + // <<- allows tab-indented delimiter; unquoted body still stripped. + cmd = "cat <<-DOC\n\tvalue > 0\n\tDOC\ntee log.txt" + if got := extractRedirectPaths(cmd); strings.Join(got, ",") != "log.txt" { + t.Fatalf("<<- heredoc: got %v", got) + } + // Unterminated heredoc swallows the rest: nothing after it counts. + if got := extractRedirectPaths("cat < not-a-file\n"); len(got) != 0 { + t.Fatalf("unterminated heredoc: got %v", got) + } +} diff --git a/internal/trace/related.go b/internal/trace/related.go new file mode 100644 index 0000000..e99d37d --- /dev/null +++ b/internal/trace/related.go @@ -0,0 +1,485 @@ +package trace + +import ( + "path/filepath" + "regexp" + "sort" + "strings" + "time" + "unicode/utf8" + + "github.com/thevibeworks/ccx/internal/parser" +) + +// Session connections (docs/design/0006-session-connections.md): the +// deterministic relations between one anchor session and the other +// sessions of its workspace, each backed by evidence a reader can walk +// to. Stated from the anchor's point of view: "anchor forked_from X", +// "anchor handoff_from X" (anchor read what X wrote). + +// RelatedSession is one session connected to the anchor. +type RelatedSession struct { + SessionID string `json:"session_id"` + Provider string `json:"provider,omitempty"` + Summary string `json:"summary,omitempty"` + Start time.Time `json:"start"` + End time.Time `json:"end"` + Strength string `json:"strength"` // strong | medium | weak + Relations []Relation `json:"relations"` +} + +// Relation is one kind of link plus its evidence. +type Relation struct { + Kind string `json:"kind"` + // Count is the total behind a sampled list: shared message uuids + // for forks, files for builds_on. Paths/Evidence are capped; + // Truncated says so. + Count int `json:"count,omitempty"` + Paths []string `json:"paths,omitempty"` + Evidence []RelationEvidence `json:"evidence,omitempty"` + Truncated bool `json:"truncated,omitempty"` +} + +// RelationEvidence is one anchor into a transcript: which session, +// which message, when, and (for file relations) which path. +type RelationEvidence struct { + SessionID string `json:"session_id"` + MessageID string `json:"message_id,omitempty"` + Time time.Time `json:"time,omitempty"` + Path string `json:"path,omitempty"` + Quote string `json:"quote,omitempty"` +} + +const ( + RelForkedFrom = "forked_from" + RelForkOf = "fork_of" + RelMentions = "mentions" + RelMentionedBy = "mentioned_by" + RelHandoffFrom = "handoff_from" + RelHandoffTo = "handoff_to" + RelBuildsOn = "builds_on" + RelBuiltOnBy = "built_on_by" + RelOverlaps = "overlaps" + RelPrevious = "previous" + RelNext = "next" + + StrengthStrong = "strong" + StrengthMedium = "medium" + StrengthWeak = "weak" + + // maxRelationPaths bounds the file list carried per relation; the + // count says how many there were. + maxRelationPaths = 5 + // idRefLen is how much of a session id must appear in text to + // count as a mention: 8 hex chars, the prefix ccx prints everywhere. + idRefLen = 8 +) + +// relationStrength is the one shared band table (0005 principle 7). +var relationStrength = map[string]string{ + RelForkedFrom: StrengthStrong, + RelForkOf: StrengthStrong, + RelMentions: StrengthStrong, + RelMentionedBy: StrengthStrong, + RelHandoffFrom: StrengthStrong, + RelHandoffTo: StrengthStrong, + RelBuildsOn: StrengthMedium, + RelBuiltOnBy: StrengthMedium, + RelOverlaps: StrengthMedium, + RelPrevious: StrengthWeak, + RelNext: StrengthWeak, +} + +var strengthRank = map[string]int{StrengthStrong: 0, StrengthMedium: 1, StrengthWeak: 2} + +// SessionProfile is what relation detection needs from one parsed +// session; building it is the only per-session cost, so callers can +// profile every session of a workspace once and relate any pair. +type SessionProfile struct { + ID string + Provider string + Summary string + Start time.Time + End time.Time + + uuids map[string]struct{} + edits map[string][]touch // path -> edits, chronological + touches map[string][]touch // path -> reads and edits, chronological + idRefs map[string]refHit // lowercase 8-hex prefix -> first message quoting it +} + +type touch struct { + msgID string + t time.Time +} + +type refHit struct { + msgID string + t time.Time + quote string +} + +var hexRunRe = regexp.MustCompile(`[0-9a-fA-F]{8,}`) + +// ProfileSession extracts the relation signals from one parsed session. +func ProfileSession(s *parser.Session) *SessionProfile { + p := &SessionProfile{ + uuids: make(map[string]struct{}), + edits: make(map[string][]touch), + touches: make(map[string][]touch), + idRefs: make(map[string]refHit), + } + if s == nil { + return p + } + p.ID = s.ID + p.Provider = s.Provider + p.Summary = s.Summary + p.Start = s.StartTime + p.End = s.EndTime + + for _, msg := range parser.FlattenSessionMessages(s) { + if msg.UUID != "" { + p.uuids[msg.UUID] = struct{}{} + } + if msg.Kind == parser.KindUserPrompt || msg.Kind == parser.KindAssistant { + for _, b := range msg.Content { + if b.Type != "text" && b.Type != "thinking" { + continue + } + p.collectIDRefs(msg, b.Text) + } + } + for _, cb := range msg.Content { + if cb.Type != "tool_use" || cb.ToolName == "" { + continue + } + paths := extractPaths(cb.ToolInput) + if len(paths) == 0 { + continue + } + isEdit := mutatingTools[cb.ToolName] + isRead := readTools[cb.ToolName] + if !isEdit && !isRead { + continue + } + for _, path := range paths { + path = absolutePath(path, s.CWD) + if path == "" { + continue + } + t := touch{msgID: msg.UUID, t: msg.Timestamp} + p.touches[path] = append(p.touches[path], t) + if isEdit { + p.edits[path] = append(p.edits[path], t) + } + } + } + } + for _, m := range []map[string][]touch{p.edits, p.touches} { + for path := range m { + list := m[path] + sort.SliceStable(list, func(i, j int) bool { return list[i].t.Before(list[j].t) }) + m[path] = list + } + } + return p +} + +func (p *SessionProfile) collectIDRefs(msg *parser.Message, text string) { + for _, loc := range hexRunRe.FindAllStringIndex(text, -1) { + prefix := strings.ToLower(text[loc[0] : loc[0]+idRefLen]) + if _, seen := p.idRefs[prefix]; seen { + continue + } + p.idRefs[prefix] = refHit{ + msgID: msg.UUID, + t: msg.Timestamp, + quote: quoteAround(text, loc[0], loc[1]-loc[0]), + } + } +} + +// absolutePath makes tool paths comparable across sessions of one +// workspace: relative paths (Codex, shell redirects) are joined onto +// the session cwd. +func absolutePath(path, cwd string) string { + path = cleanEvidencePath(path) + if path == "" { + return "" + } + if !filepath.IsAbs(path) && cwd != "" { + path = filepath.Join(cwd, path) + } + return filepath.Clean(path) +} + +// isBatonPath reports whether a path is a session baton — the files +// one session writes so the next can pick the work up. +func isBatonPath(path string) bool { + lower := strings.ToLower(filepath.ToSlash(path)) + base := filepath.Base(lower) + switch { + case strings.HasPrefix(base, "handoff") && (strings.HasSuffix(base, ".md") || strings.HasSuffix(base, ".org")): + return true + case strings.Contains(lower, "/handoffs/"): + return true + case strings.Contains(lower, "devlog"): + return true + case base == "plan.md", base == "todo.md", base == "notes.md": + return true + } + return false +} + +// idPrefix is the lowercase 8-hex prefix ccx prints for a session id; +// empty when the id does not start with one. +func idPrefix(id string) string { + id = strings.ToLower(id) + if len(id) < idRefLen { + return "" + } + for i := 0; i < idRefLen; i++ { + c := id[i] + if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f')) { + return "" + } + } + return id[:idRefLen] +} + +// RelateSessions computes the anchor's connections to others (any +// order; the anchor itself is skipped if present). Result is strongest +// first, then by start time. +func RelateSessions(anchor *SessionProfile, others []*SessionProfile) []RelatedSession { + if anchor == nil { + return nil + } + var out []RelatedSession + var prev, next *SessionProfile + for _, o := range others { + if o == nil || o.ID == anchor.ID { + continue + } + if !o.Start.IsZero() && !anchor.Start.IsZero() { + if o.Start.Before(anchor.Start) && (prev == nil || o.Start.After(prev.Start)) { + prev = o + } + if o.Start.After(anchor.Start) && (next == nil || o.Start.Before(next.Start)) { + next = o + } + } + } + for _, o := range others { + if o == nil || o.ID == anchor.ID { + continue + } + rels := relatePair(anchor, o) + if o == prev { + rels = append(rels, Relation{Kind: RelPrevious}) + } + if o == next { + rels = append(rels, Relation{Kind: RelNext}) + } + if len(rels) == 0 { + continue + } + out = append(out, RelatedSession{ + SessionID: o.ID, + Provider: o.Provider, + Summary: o.Summary, + Start: o.Start, + End: o.End, + Strength: strongest(rels), + Relations: rels, + }) + } + sort.SliceStable(out, func(i, j int) bool { + a, b := out[i], out[j] + if strengthRank[a.Strength] != strengthRank[b.Strength] { + return strengthRank[a.Strength] < strengthRank[b.Strength] + } + return a.Start.Before(b.Start) + }) + return out +} + +func strongest(rels []Relation) string { + best := StrengthWeak + for _, r := range rels { + if s := relationStrength[r.Kind]; strengthRank[s] < strengthRank[best] { + best = s + } + } + return best +} + +// relatePair finds every relation between anchor a and other o, from +// a's point of view. +func relatePair(a, o *SessionProfile) []Relation { + var rels []Relation + + // Fork: shared message uuids; the earlier session is the origin. + if shared, first := sharedUUIDs(a, o); shared > 0 { + kind := RelForkOf + if !a.Start.IsZero() && !o.Start.IsZero() && a.Start.After(o.Start) { + kind = RelForkedFrom + } + rels = append(rels, Relation{ + Kind: kind, + Count: shared, + Evidence: []RelationEvidence{{SessionID: a.ID, MessageID: first}, {SessionID: o.ID, MessageID: first}}, + }) + } + + // Mentions: one session's text names the other's id. + if pfx := idPrefix(o.ID); pfx != "" { + if hit, ok := a.idRefs[pfx]; ok { + rels = append(rels, Relation{Kind: RelMentions, Evidence: []RelationEvidence{ + {SessionID: a.ID, MessageID: hit.msgID, Time: hit.t, Quote: hit.quote}, + }}) + } + } + if pfx := idPrefix(a.ID); pfx != "" { + if hit, ok := o.idRefs[pfx]; ok { + rels = append(rels, Relation{Kind: RelMentionedBy, Evidence: []RelationEvidence{ + {SessionID: o.ID, MessageID: hit.msgID, Time: hit.t, Quote: hit.quote}, + }}) + } + } + + // Files: o wrote, a touched later -> a handoff_from / builds_on o; + // a wrote, o touched later -> a handoff_to / built_on_by o. + if h, b := fileLinks(o, a); h != nil || b != nil { + if h != nil { + h.Kind = RelHandoffFrom + rels = append(rels, *h) + } + if b != nil { + b.Kind = RelBuildsOn + rels = append(rels, *b) + } + } + if h, b := fileLinks(a, o); h != nil || b != nil { + if h != nil { + h.Kind = RelHandoffTo + rels = append(rels, *h) + } + if b != nil { + b.Kind = RelBuiltOnBy + rels = append(rels, *b) + } + } + + // Overlap: concurrent windows. + if !a.Start.IsZero() && !a.End.IsZero() && !o.Start.IsZero() && !o.End.IsZero() && + a.Start.Before(o.End) && o.Start.Before(a.End) { + start, end := a.Start, a.End + if o.Start.After(start) { + start = o.Start + } + if o.End.Before(end) { + end = o.End + } + rels = append(rels, Relation{Kind: RelOverlaps, Evidence: []RelationEvidence{ + {SessionID: a.ID, Time: start}, {SessionID: o.ID, Time: end}, + }}) + } + return rels +} + +func sharedUUIDs(a, o *SessionProfile) (int, string) { + small, large := a.uuids, o.uuids + if len(large) < len(small) { + small, large = large, small + } + n := 0 + first := "" + for id := range small { + if _, ok := large[id]; ok { + n++ + if first == "" || id < first { + first = id + } + } + } + return n, first +} + +// fileLinks finds paths writer edited that reader touched afterwards, +// split into baton files (handoff) and everything else (builds_on). +// Evidence pairs the writer's first qualifying edit with the reader's +// first touch after it. Paths are sorted for deterministic output. +func fileLinks(writer, reader *SessionProfile) (handoff, builds *Relation) { + paths := make([]string, 0, len(writer.edits)) + for path := range writer.edits { + if _, ok := reader.touches[path]; ok { + paths = append(paths, path) + } + } + sort.Strings(paths) + + add := func(rel **Relation, path string, w, r touch) { + if *rel == nil { + *rel = &Relation{} + } + (*rel).Count++ + if len((*rel).Paths) >= maxRelationPaths { + (*rel).Truncated = true + return + } + (*rel).Paths = append((*rel).Paths, path) + (*rel).Evidence = append((*rel).Evidence, + RelationEvidence{SessionID: writer.ID, MessageID: w.msgID, Time: w.t, Path: path}, + RelationEvidence{SessionID: reader.ID, MessageID: r.msgID, Time: r.t, Path: path}, + ) + } + + for _, path := range paths { + w := writer.edits[path][0] + var r touch + found := false + for _, t := range reader.touches[path] { + if t.t.After(w.t) { + r, found = t, true + break + } + } + if !found { + continue + } + if isBatonPath(path) { + add(&handoff, path, w, r) + } else { + add(&builds, path, w, r) + } + } + return handoff, builds +} + +// quoteAround cuts a bounded window around a match, clamped to rune +// boundaries, for evidence quotes. +func quoteAround(text string, idx, n int) string { + start := idx - 40 + if start < 0 { + start = 0 + } + end := idx + n + 40 + if end > len(text) { + end = len(text) + } + for start > 0 && !utf8.RuneStart(text[start]) { + start-- + } + for end < len(text) && !utf8.RuneStart(text[end]) { + end++ + } + out := strings.Join(strings.Fields(text[start:end]), " ") + if start > 0 { + out = "..." + out + } + if end < len(text) { + out += "..." + } + return out +} diff --git a/internal/trace/related_test.go b/internal/trace/related_test.go new file mode 100644 index 0000000..7607ecc --- /dev/null +++ b/internal/trace/related_test.go @@ -0,0 +1,227 @@ +package trace + +import ( + "testing" + "time" + + "github.com/thevibeworks/ccx/internal/parser" +) + +func relT(min int) time.Time { return time.Date(2026, 8, 18, 10, min, 0, 0, time.UTC) } + +func relSession(id string, start, end time.Time, msgs ...*parser.Message) *parser.Session { + return &parser.Session{ID: id, Provider: "claude-code", CWD: "/w", StartTime: start, EndTime: end, RootMessages: msgs} +} + +func relUser(uuid string, t time.Time, text string) *parser.Message { + return &parser.Message{UUID: uuid, Type: "user", Kind: parser.KindUserPrompt, Timestamp: t, + Content: []parser.ContentBlock{{Type: "text", Text: text}}} +} + +func relTool(uuid string, t time.Time, tool string, input map[string]any) *parser.Message { + return &parser.Message{UUID: uuid, Type: "assistant", Kind: parser.KindAssistant, Timestamp: t, + Content: []parser.ContentBlock{{Type: "tool_use", ToolName: tool, ToolID: "t-" + uuid, ToolInput: input}}} +} + +func kinds(rels []Relation) map[string]Relation { + m := make(map[string]Relation) + for _, r := range rels { + m[r.Kind] = r + } + return m +} + +func findRelated(list []RelatedSession, id string) *RelatedSession { + for i := range list { + if list[i].SessionID == id { + return &list[i] + } + } + return nil +} + +// A handoff is a baton file written by one session and read by a +// later one; an ordinary file edited then read is builds_on; both +// carry writer and reader anchors. Direction flips with the point of +// view, and a read that happened BEFORE the write is not a link. +func TestRelateSessionsHandoffAndBuildsOn(t *testing.T) { + writer := relSession("aaaaaaaa-1", relT(0), relT(10), + relTool("w1", relT(1), "Write", map[string]any{"file_path": "/w/HANDOFF.md"}), + relTool("w2", relT(2), "Edit", map[string]any{"file_path": "/w/src/main.go"}), + relTool("w3", relT(3), "Read", map[string]any{"file_path": "/w/README.md"}), + ) + reader := relSession("bbbbbbbb-2", relT(20), relT(30), + relTool("r1", relT(21), "Read", map[string]any{"file_path": "/w/HANDOFF.md"}), + relTool("r2", relT(22), "Read", map[string]any{"file_path": "src/main.go"}), // relative: joined onto cwd + relTool("r3", relT(23), "Edit", map[string]any{"file_path": "/w/README.md"}), + ) + pw, pr := ProfileSession(writer), ProfileSession(reader) + + got := RelateSessions(pr, []*SessionProfile{pw}) + if len(got) != 1 || got[0].SessionID != "aaaaaaaa-1" || got[0].Strength != StrengthStrong { + t.Fatalf("reader view: %+v", got) + } + k := kinds(got[0].Relations) + h, ok := k[RelHandoffFrom] + if !ok || h.Count != 1 || h.Paths[0] != "/w/HANDOFF.md" || len(h.Evidence) != 2 { + t.Fatalf("handoff_from: %+v", h) + } + if h.Evidence[0].SessionID != "aaaaaaaa-1" || h.Evidence[0].MessageID != "w1" || h.Evidence[1].MessageID != "r1" { + t.Fatalf("handoff evidence must pair writer then reader: %+v", h.Evidence) + } + b, ok := k[RelBuildsOn] + if !ok || b.Count != 1 || b.Paths[0] != "/w/src/main.go" { + t.Fatalf("builds_on: %+v (README was read by writer, not edited — must not count)", b) + } + if _, ok := k[RelBuiltOnBy]; ok { + t.Fatal("reader edited README after writer READ it; that is not built_on_by") + } + if _, ok := k[RelPrevious]; !ok { + t.Fatal("writer is the reader's nearest earlier session") + } + + got = RelateSessions(pw, []*SessionProfile{pr}) + k = kinds(got[0].Relations) + if _, ok := k[RelHandoffTo]; !ok { + t.Fatalf("writer view must say handoff_to: %+v", k) + } + if _, ok := k[RelBuiltOnBy]; !ok { + t.Fatalf("writer view must say built_on_by: %+v", k) + } + if _, ok := k[RelNext]; !ok { + t.Fatal("reader is the writer's next session") + } +} + +// Shared message uuids mean a fork; the earlier session is the origin. +// A mention is the other session's 8-hex prefix in conversation text +// (not in tool results), quoted as evidence. +func TestRelateSessionsForkAndMentions(t *testing.T) { + origin := relSession("11111111-aaaa", relT(0), relT(5), + relUser("shared-1", relT(0), "start"), + relUser("shared-2", relT(1), "more"), + ) + fork := relSession("22222222-bbbb", relT(10), relT(15), + relUser("shared-1", relT(0), "start"), + relUser("shared-2", relT(1), "more"), + relUser("own-3", relT(11), "continue from session 11111111 please"), + ) + // A tool result naming the id is not a mention. + bystander := relSession("33333333-cccc", relT(20), relT(25), + &parser.Message{UUID: "tr", Type: "user", Kind: parser.KindToolResult, Timestamp: relT(21), + Content: []parser.ContentBlock{{Type: "tool_result", ToolResult: "22222222-bbbb"}, {Type: "text", Text: "session 22222222 listed"}}}, + ) + po, pf, pb := ProfileSession(origin), ProfileSession(fork), ProfileSession(bystander) + + got := RelateSessions(pf, []*SessionProfile{po, pb}) + r := findRelated(got, "11111111-aaaa") + if r == nil || r.Strength != StrengthStrong { + t.Fatalf("fork view of origin: %+v", got) + } + k := kinds(r.Relations) + f, ok := k[RelForkedFrom] + if !ok || f.Count != 2 || f.Evidence[0].MessageID != "shared-1" { + t.Fatalf("forked_from: %+v", f) + } + m, ok := k[RelMentions] + if !ok || m.Evidence[0].MessageID != "own-3" || m.Evidence[0].SessionID != "22222222-bbbb" || m.Evidence[0].Quote == "" { + t.Fatalf("mentions: %+v", m) + } + if b := findRelated(got, "33333333-cccc"); b != nil { + for _, rel := range b.Relations { + if rel.Kind == RelMentionedBy { + t.Fatalf("tool-result text must not count as a mention: %+v", rel) + } + } + } + + got = RelateSessions(po, []*SessionProfile{pf}) + k = kinds(got[0].Relations) + if _, ok := k[RelForkOf]; !ok { + t.Fatalf("origin view must say fork_of: %+v", k) + } + if _, ok := k[RelMentionedBy]; !ok { + t.Fatalf("origin view must say mentioned_by: %+v", k) + } +} + +// Overlap needs intersecting windows; ordering is strongest first, +// then by start; unrelated sessions are absent; the anchor is skipped. +func TestRelateSessionsOverlapOrderingAndSelf(t *testing.T) { + anchor := relSession("aaaaaaaa-0", relT(10), relT(20), + relTool("a1", relT(12), "Edit", map[string]any{"file_path": "/w/x.go"})) + concurrent := relSession("bbbbbbbb-1", relT(15), relT(25)) + later := relSession("cccccccc-2", relT(30), relT(35), + relTool("c1", relT(31), "Read", map[string]any{"file_path": "/w/x.go"})) + unrelated := relSession("dddddddd-3", relT(40), relT(45), + relTool("d1", relT(41), "Read", map[string]any{"file_path": "/w/other.go"})) + pa := ProfileSession(anchor) + got := RelateSessions(pa, []*SessionProfile{ProfileSession(unrelated), ProfileSession(later), ProfileSession(concurrent), pa}) + + if len(got) != 2 { + t.Fatalf("want concurrent + later only, got %+v", got) + } + // Both medium: order by start. + if got[0].SessionID != "bbbbbbbb-1" || got[1].SessionID != "cccccccc-2" { + t.Fatalf("order: %s, %s", got[0].SessionID, got[1].SessionID) + } + k := kinds(got[0].Relations) + o, ok := k[RelOverlaps] + if !ok || !o.Evidence[0].Time.Equal(relT(15)) || !o.Evidence[1].Time.Equal(relT(20)) { + t.Fatalf("overlap window: %+v", o) + } + if _, ok := kinds(got[1].Relations)[RelBuiltOnBy]; !ok { + t.Fatalf("later read of anchor's edit: %+v", got[1].Relations) + } + // Timeless sessions never become previous/next. + if got := RelateSessions(pa, []*SessionProfile{ProfileSession(relSession("eeeeeeee-4", time.Time{}, time.Time{}))}); len(got) != 0 { + t.Fatalf("timeless session related: %+v", got) + } +} + +// Path lists are capped at maxRelationPaths with the count intact and +// Truncated set — never a silent cut. +func TestRelateSessionsCapsPaths(t *testing.T) { + var wmsgs, rmsgs []*parser.Message + for i := 0; i < maxRelationPaths+3; i++ { + p := "/w/f" + string(rune('a'+i)) + ".go" + wmsgs = append(wmsgs, relTool("w"+string(rune('a'+i)), relT(1), "Edit", map[string]any{"file_path": p})) + rmsgs = append(rmsgs, relTool("r"+string(rune('a'+i)), relT(30), "Read", map[string]any{"file_path": p})) + } + pw := ProfileSession(relSession("aaaaaaaa-w", relT(0), relT(5), wmsgs...)) + pr := ProfileSession(relSession("bbbbbbbb-r", relT(20), relT(40), rmsgs...)) + got := RelateSessions(pr, []*SessionProfile{pw}) + b := kinds(got[0].Relations)[RelBuildsOn] + if b.Count != maxRelationPaths+3 || len(b.Paths) != maxRelationPaths || !b.Truncated { + t.Fatalf("cap: count=%d paths=%d truncated=%v", b.Count, len(b.Paths), b.Truncated) + } +} + +func TestIsBatonPath(t *testing.T) { + yes := []string{"/w/HANDOFF.md", "/w/handoff-notes.org", "/w/.claude/handoffs/latest.md", "/w/docs/devlog/2026-08-18-x.org", "/w/DEVLOG.org", "/w/PLAN.md", "/w/todo.md"} + no := []string{"/w/README.md", "/w/internal/cmd/search.go", "/w/handoff.go", "/w/docs/design/0006.md"} + for _, p := range yes { + if !isBatonPath(p) { + t.Errorf("%s should be a baton path", p) + } + } + for _, p := range no { + if isBatonPath(p) { + t.Errorf("%s should not be a baton path", p) + } + } +} + +func TestIDPrefix(t *testing.T) { + cases := map[string]string{ + "736a7bac-0a5d-4e3f-9036-c8a94111a347": "736a7bac", + "019F6528-ABCD": "019f6528", + "short": "", + "zzzzzzzz-not-hex": "", + } + for in, want := range cases { + if got := idPrefix(in); got != want { + t.Errorf("idPrefix(%q) = %q, want %q", in, got, want) + } + } +} diff --git a/internal/trace/types.go b/internal/trace/types.go index 7adbe08..a4febc0 100644 --- a/internal/trace/types.go +++ b/internal/trace/types.go @@ -20,8 +20,13 @@ type TraceResult struct { Sidechains []Sidechain `json:"sidechains,omitempty"` Git GitCorrelation `json:"git"` Workspace WorkspaceContext `json:"workspace_context"` - Stats TraceStats `json:"stats"` - Warnings []TraceWarning `json:"warnings,omitempty"` + // Related is the session's connections to the other sessions of + // its workspace (docs/design/0006-session-connections.md); filled + // only for the full bundle, since it costs a parse of every + // session in the workspace. + Related []RelatedSession `json:"related,omitempty"` + Stats TraceStats `json:"stats"` + Warnings []TraceWarning `json:"warnings,omitempty"` } type SessionMeta struct { diff --git a/skills/ccx/SKILL.md b/skills/ccx/SKILL.md index 0cb35f3..1932e7b 100644 --- a/skills/ccx/SKILL.md +++ b/skills/ccx/SKILL.md @@ -36,7 +36,9 @@ ccx ├── trace [session] # What the agent did: turn/step outline │ └── --json # Outline as JSON (ccx.outline.v1) │ └── --turn N # Full evidence for one turn (ccx.turn.v1) -│ └── --full # Complete trace bundle (ccx.trace.v2, large) +│ └── --full # Complete trace bundle (ccx.trace.v2, large; includes related) +├── related [session] # Which sessions connect to this one, and how +│ └── --json # Every relation with evidence (ccx.related.v1) ├── log [project] # Slice raw session logs by time scope │ └── --scope today|yesterday|week|month|quarter|year │ └── --since / --until TIME # RFC3339 or YYYY-MM-DD @@ -72,6 +74,7 @@ ccx sessions --scope yesterday --tz +8 --all --json # Session containers by end ccx log --scope yesterday --tz +8 --all --json ccx trace # Outline of the latest workspace session ccx trace abc123 --turn 5 # Full evidence for one turn +ccx related abc123 # Sessions connected to this one: fork, handoff, mentions, shared files ccx web # Start web UI at localhost:8080 ``` @@ -86,6 +89,19 @@ complete bundle. JSON kinds and field semantics are documented in JSON times are UTC. Session IDs resolve across all projects automatically; `ccx trace ` works from any directory. +## Related: connections between sessions + +`ccx related [session]` joins the islands: which other sessions of the +workspace connect to this one and how — `forked_from`/`fork_of` +(shared message ids), `mentions`/`mentioned_by` (a session id named +in text), `handoff_from`/`handoff_to` (a baton file such as HANDOFF.md, +handoffs/, devlog written by one and read by the other later), +`builds_on`/`built_on_by` (a file edited by one, then touched by the +other), `overlaps` (concurrent), `previous`/`next`. Strength is a band +(strong/medium/weak). `--json` carries the evidence per relation +(session, message id, time, path, quote) — cite from that when a +claim spans sessions. Also present as `related` in `ccx trace --full`. + ## Multi-Provider ```bash From afa378513d45942d26ae2d7b0f5a10a85268b050 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 05:55:11 -0700 Subject: [PATCH 04/13] fix(log): conversation-only kinds, Codex 0.147 items; add --kind/--match MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ccx log` is the time-sliced evidence layer, but it disagreed with the parser about what a human said: - Claude user-role lines that no human typed — slash-command markers, echoes, task notifications, injected meta (skill bodies), compaction carriers — were all `user_prompt`. Today's slice reported 233 prompts; 79 were typed by a person. sessionlog now reuses parser.ClassifyUserText (exported) plus isMeta / isCompactSummary, so kinds match the full parser: command, command_output, notification, meta, compact_summary. - Codex 0.147 rollouts rendered the real conversation (event_msg.item_completed UserMessage/AgentMessage) as bare `item_completed` rows with no text, while raw response_item messages — including the injected AGENTS.md envelope — showed as `user_prompt`. sessionlog now decodes TurnItems via codex.DecodeCompletedTurnMessage (exported from turn_items.go) and applies the parser's one-source-per-rollout rule (docs/design/0004): in a rollout with completed items, legacy events become `legacy_message` and raw response_item messages `model_input`/`model_output`. Nothing is dropped; session kinds and preview are re-tallied so metrics count the conversation once. With the kinds trustworthy, two filters make the firehose a timeline: `--kind K1,K2` keeps record kinds; `--match PHRASE [-w]` keeps records whose raw transcript line contains the phrase (grep parity, the time-bounded complement of `search --hits`). `metrics.records` stays scope-wide, `records_matched` is the narrowed count. ccx log --scope today --all --kind user_prompt # the humans in the loop ccx log --scope month --all --match deadman -w # when a term came up Tests: TestCollectCodex0147ItemCompletedIsTheConversation, TestCollectCodexLegacyRolloutUnchanged, TestCollectClaudeUserRoleNoiseIsNotAPrompt (kinds, metrics, --kind, --match on the raw line). Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 2 + docs/schema.md | 2 +- internal/cmd/log.go | 35 ++++- internal/parser/classify_test.go | 4 +- internal/parser/session.go | 12 +- internal/provider/codex/backend.go | 4 +- internal/provider/codex/turn_items.go | 21 ++- internal/sessionlog/sessionlog.go | 173 +++++++++++++++++++++++-- internal/sessionlog/sessionlog_test.go | 146 +++++++++++++++++++++ skills/ccx/SKILL.md | 4 + 10 files changed, 370 insertions(+), 33 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e3afa6c..e92db3a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,9 +14,11 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) - **`ccx search --content` reports when: `FIRST` column, `first_hit` in `--json`, `--sort first|last|hits`.** "When did we first mention X" needs the earliest matching message and an oldest-first order; results only carried session end time and ranked by hit count. Each content hit now records the timestamp of its earliest matching message (parsed messages by default; the raw line's top-level `timestamp` under `--raw`), printed as `FIRST` and sortable with `--sort first`; `--sort last` orders by session activity; `--sort hits` is the old order and the default. ### Changed +- **`ccx log --kind` and `--match PHRASE [-w]` turn the firehose into a timeline.** `log` emitted every record in scope (16k for one day) with no way to narrow; "what did the humans ask today, across every session" was not answerable. `--kind user_prompt,assistant_message` keeps only those kinds; `--match` keeps records whose raw transcript line contains the phrase (grep parity, `-w` for whole words) — the time-bounded complement of `search --hits`. Metrics stay honest: `records` is scope-wide, new `records_matched` is the narrowed count, `showing` is after `-n`. - **`ccx search --content` is ~10x faster and shows progress.** The scan ran on one core and lowercased every transcript line: 6m18s cold / 1m14s warm over a 3.5 GB store, silent throughout. Sessions now scan on a bounded worker pool (up to 8), the raw prefilter matches case-insensitively without allocating and stops at the first hit, and a `scanning N/M sessions` line ticks on stderr when it is a terminal. Same store, warm: 7.6s. ### Fixed +- **`ccx log` now applies the same conversation rules as the parser.** Two classes of user-role lines were reported as `user_prompt`: Claude harness wrappers (slash-command markers, `` echoes, task notifications), injected meta messages (skill bodies), and compaction carriers — 233 "prompts" today of which 79 were typed by a human; and Codex 0.147 raw `response_item` messages, including the injected AGENTS.md envelope, while the real `item_completed` UserMessage/AgentMessage rows rendered as bare `item_completed` with no text. `sessionlog` now reuses `parser.ClassifyUserText` and the Codex TurnItem decoder (`DecodeCompletedTurnMessage`, exported), demotes legacy/raw duplicates in a 0.147 rollout to `legacy_message`/`model_input`/`model_output` (records kept, counts fixed), and re-tallies session kinds and preview after demotion. `user_prompts`/`assistant_messages` metrics and `insight` reports built on them count the conversation once. - **Heredoc bodies no longer count as shell redirects.** `extractRedirectPaths` scanned the whole Bash command, so a Go `if n > 0` or a markdown `> 2026-08-18` inside `python3 - <<'EOF'` / `cat > f <<'EOF'` became "edited files" (`.../0`, `.../2026-08-18`) in `trace` `files_edited` and in session connections. Heredoc bodies are stripped before the redirect scan. - **`-n` is the `--limit` shorthand everywhere.** Only `search` had it; `sessions -n 2` failed with "unknown shorthand flag". `sessions`, `projects`, and `log` now accept `-n` too. - **A session whose summary matched dropped its content evidence.** The summary hit short-circuited the content scan, so under `--content` the session where the term was actually discussed could be the one result with no hit count, previews, or first-hit time. Summary hits stay typed `session` but now carry the content fields. diff --git a/docs/schema.md b/docs/schema.md index b3b30aa..bedf37c 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -16,7 +16,7 @@ they do not know. | `ccx.turn.v1` | `ccx trace --turn N` | One turn with full step evidence, plus the sidechain entries that turn references, plus warnings. | | `ccx.trace.v2` | `ccx trace --full` | Complete evidence bundle: all turns/steps, sidechains, git correlation, workspace context, `related` sessions, stats, warnings. Large. | | `ccx.related.v1` | `ccx related --json` | The anchor session's connections to the other sessions of its workspace: `related[]` of `{session_id, provider, summary, start, end, strength, relations[]}`, plus `total`/`shown`. Each relation is `{kind, count?, paths?, evidence[], truncated?}`; evidence items are `{session_id, message_id, time, path?, quote?}`. Kinds: `forked_from`/`fork_of`, `mentions`/`mentioned_by`, `handoff_from`/`handoff_to`, `builds_on`/`built_on_by`, `overlaps`, `previous`/`next`. Strength is `strong`/`medium`/`weak`. | -| `ccx.log.v1` | `ccx log --json`, `ccx insight --json` | Time-scoped records across sessions with pre-computed `days[]` / `providers[]` / `workspaces[]` aggregates. | +| `ccx.log.v1` | `ccx log --json`, `ccx insight --json` | Time-scoped records across sessions with pre-computed `days[]` / `providers[]` / `workspaces[]` aggregates. Record `kind` is provider-normalized: `user_prompt` and `assistant_message` are the visible conversation only (Claude command markers/echoes/notifications are `command`/`command_output`/`notification`, injected meta is `meta`, compaction carriers `compact_summary`; Codex 0.147 raw `response_item` messages are `model_input`/`model_output` and duplicated legacy events `legacy_message`). With `--kind`/`--match`, `metrics.records` stays scope-wide and `metrics.records_matched` reports the narrowed count before any limit. | ## Versioning policy diff --git a/internal/cmd/log.go b/internal/cmd/log.go index 9e0ebcd..357ccaf 100644 --- a/internal/cmd/log.go +++ b/internal/cmd/log.go @@ -24,11 +24,19 @@ Sessions can run for days or months, so time-scoped review must use log records, not just session end times. This command emits the evidence layer for scoped insight: records inside the window plus session overlap metadata. +--kind narrows to record kinds (comma-separated: user_prompt, +assistant_message, tool_call, tool_result, reasoning, ...); --match +keeps records whose raw transcript line contains a phrase (-w for +whole words). Together they turn the firehose into a timeline: +every human prompt today, across every session, in order. + Examples: ccx log --scope yesterday --tz +8 --all --json ccx log --since 2026-05-21 --until 2026-05-22 --tz +8 --all --json ccx log --scope week --provider cx --json - ccx log /path/to/repo --scope yesterday --json --raw`, + ccx log /path/to/repo --scope yesterday --json --raw + ccx log --scope today --all --kind user_prompt # the humans in the loop, today + ccx log --scope month --all --match semantica -w # when a term came up, in a window`, Args: cobra.MaximumNArgs(1), RunE: runLog, } @@ -43,6 +51,9 @@ var ( logAll bool logProvider string logLimit int + logKinds string + logMatch string + logWord bool logLocation *time.Location ) @@ -56,6 +67,9 @@ func init() { logCmd.Flags().BoolVar(&logAll, "all", false, "slice logs across all projects") logCmd.Flags().StringVarP(&logProvider, "provider", "p", "", "filter by provider: cc, cx, all") logCmd.Flags().IntVarP(&logLimit, "limit", "n", 0, "limit records in JSON output (0 = no limit)") + logCmd.Flags().StringVar(&logKinds, "kind", "", "keep only these record kinds (comma-separated, e.g. user_prompt,assistant_message)") + logCmd.Flags().StringVar(&logMatch, "match", "", "keep only records whose raw transcript line contains this phrase") + logCmd.Flags().BoolVarP(&logWord, "word", "w", false, "--match whole words only") } func runLog(cmd *cobra.Command, args []string) error { @@ -86,6 +100,16 @@ func runLog(cmd *cobra.Command, args []string) error { return fmt.Errorf("invalid --provider %q (want cc, cx, claude-code, codex, or all)", logProvider) } + var kinds []string + if strings.TrimSpace(logKinds) != "" { + kinds = strings.Split(logKinds, ",") + } + var match func(string) bool + if strings.TrimSpace(logMatch) != "" { + m := newTextMatcher(logMatch, logWord) + match = m.matches + } + settings := config.Load() bundle, err := sessionlog.Collect(logSources(settings, providerFilter), sessionlog.Options{ Start: start, @@ -99,6 +123,8 @@ func runLog(cmd *cobra.Command, args []string) error { Limit: logLimit, IncludeRaw: logRaw, Now: time.Now().In(loc), + Kinds: kinds, + Match: match, }) if err != nil { return err @@ -178,8 +204,11 @@ func printLogTable(bundle *sessionlog.Bundle) error { } fmt.Printf("ccx log · %s · %s\n", bundle.Scope.Label, bundle.Scope.TimeZone) recordsLabel := fmt.Sprintf("records %d", bundle.Metrics.Records) - if bundle.Metrics.RecordsReturned != bundle.Metrics.Records { - recordsLabel = fmt.Sprintf("records %d · showing %d", bundle.Metrics.Records, bundle.Metrics.RecordsReturned) + if bundle.Metrics.RecordsMatched > 0 || len(bundle.Records) == 0 && bundle.Metrics.Records > 0 && bundle.Metrics.RecordsReturned == 0 { + recordsLabel = fmt.Sprintf("records %d · matched %d", bundle.Metrics.Records, bundle.Metrics.RecordsMatched) + } + if bundle.Metrics.RecordsReturned != bundle.Metrics.Records && bundle.Metrics.RecordsReturned != bundle.Metrics.RecordsMatched { + recordsLabel += fmt.Sprintf(" · showing %d", bundle.Metrics.RecordsReturned) } fmt.Printf("%s to %s · source log files %d · %s\n\n", bundle.Scope.Start.Format("2006-01-02 15:04"), diff --git a/internal/parser/classify_test.go b/internal/parser/classify_test.go index 379b079..27791bd 100644 --- a/internal/parser/classify_test.go +++ b/internal/parser/classify_test.go @@ -18,9 +18,9 @@ func TestClassifyUserTextHarnessWrappers(t *testing.T) { {"", KindUnknown, false}, } for _, c := range cases { - got, harness := classifyUserText(c.text) + got, harness := ClassifyUserText(c.text) if harness != c.harness || (harness && got != c.want) { - t.Errorf("classifyUserText(%q) = (%v, %v), want (%v, %v)", c.text, got, harness, c.want, c.harness) + t.Errorf("ClassifyUserText(%q) = (%v, %v), want (%v, %v)", c.text, got, harness, c.want, c.harness) } } } diff --git a/internal/parser/session.go b/internal/parser/session.go index 170c20a..4a99cd2 100644 --- a/internal/parser/session.go +++ b/internal/parser/session.go @@ -256,11 +256,11 @@ func classifyMessage(msg *Message, raw rawMessage) MessageKind { if len(msg.Content) > 0 && msg.Content[0].Type == "text" { text = msg.Content[0].Text } - kind, ok := classifyUserText(text) + kind, ok := ClassifyUserText(text) if !ok { if str, isStr := raw.Message.Content.(string); isStr { text = str - kind, ok = classifyUserText(text) + kind, ok = ClassifyUserText(text) } } if ok { @@ -279,11 +279,13 @@ func classifyMessage(msg *Message, raw rawMessage) MessageKind { return KindUnknown } -// classifyUserText recognizes harness-generated XML wrappers in a +// ClassifyUserText recognizes harness-generated XML wrappers in a // user-role message. These all masquerade as human turns in the raw // JSONL but carry no human input: slash-command markers, local command // stdout/stderr/caveat echoes, and background-task notifications. -func classifyUserText(text string) (MessageKind, bool) { +// Exported so line-streaming surfaces (sessionlog) classify a +// user-role line exactly as the full parser does. +func ClassifyUserText(text string) (MessageKind, bool) { t := strings.TrimSpace(text) switch { case strings.HasPrefix(t, " 0 || opts.Match != nil + if filtered { + bundle.Records = filterRecords(bundle.Records, opts.Kinds) + } + matched := len(bundle.Records) bundle.Days, bundle.Providers, bundle.Workspaces = aggregateRecords(bundle.Records, now.Location()) if opts.Limit > 0 && len(bundle.Records) > opts.Limit { bundle.Records = bundle.Records[:opts.Limit] } bundle.Metrics = metricsFor(bundle.Sessions, len(bundle.Records), opts.Limit, opts.Start, opts.End) + if filtered { + bundle.Metrics.RecordsMatched = matched + } return bundle, nil } +// filterRecords keeps records that passed Match and are of one of the +// kinds (kind demotion has already run, so "user_prompt" means the +// visible conversation on every provider). +func filterRecords(records []Record, kinds []string) []Record { + want := make(map[string]bool, len(kinds)) + for _, k := range kinds { + if k = strings.TrimSpace(k); k != "" { + want[k] = true + } + } + out := records[:0] + for _, r := range records { + if !r.matched { + continue + } + if len(want) > 0 && !want[r.Kind] { + continue + } + out = append(out, r) + } + return out +} + // aggregateRecords buckets records by calendar day (in loc), provider, // and workspace. Days sort chronologically; providers and workspaces // sort by record volume, busiest first. @@ -441,6 +486,7 @@ func scanFile(source Source, filePath string, opts Options) (*fileScan, error) { if opts.IncludeRaw { record.RawJSON = json.RawMessage([]byte(line)) } + record.matched = opts.Match == nil || opts.Match(line) scan.Records = append(scan.Records, record) scan.Kinds[record.Kind]++ if scan.Preview == "" && record.Text != "" && (record.Kind == "user_prompt" || record.Kind == "assistant_message") { @@ -453,6 +499,19 @@ func scanFile(source Source, filePath string, opts Options) (*fileScan, error) { if len(scan.Records) == 0 { return nil, nil } + if source.Provider == "codex" && demoteLegacyCodexMessages(scan.Records) { + // Kinds and preview were tallied while streaming; redo them + // against the demoted kinds so metrics count the conversation + // once and the preview is a visible prompt, not an envelope. + scan.Kinds = make(map[string]int) + scan.Preview = "" + for _, record := range scan.Records { + scan.Kinds[record.Kind]++ + if scan.Preview == "" && record.Text != "" && (record.Kind == "user_prompt" || record.Kind == "assistant_message") { + scan.Preview = record.Text + } + } + } for i := range scan.Records { if (scan.Records[i].SessionID == "" || scan.Records[i].SessionID == fallbackSessionID(filePath)) && scan.SessionID != "" { scan.Records[i].SessionID = scan.SessionID @@ -550,10 +609,26 @@ func normalizeClaudeRecord(raw rawLine) Record { switch raw.Type { case "user": - if contentHasType(content, "tool_result") { + // Same rules as the full parser (parser.classifyMessage): a + // user-role line is a human prompt only when it is not a + // compaction carrier, an injected meta message (skill bodies, + // system instructions), a tool result, or a harness XML + // wrapper (slash-command markers, local command echoes, + // task notifications). Otherwise "the humans in the loop" + // shows things no human typed. + switch { + case raw.IsCompactSummary: + record.Kind = "compact_summary" + case raw.IsMeta: + record.Kind = "meta" + case contentHasType(content, "tool_result"): record.Kind = "tool_result" - } else { - record.Kind = "user_prompt" + default: + if kind, ok := parser.ClassifyUserText(userTextForClassify(content)); ok { + record.Kind = string(kind) + } else { + record.Kind = "user_prompt" + } } case "assistant": if contentHasType(content, "tool_use") { @@ -572,6 +647,28 @@ func normalizeClaudeRecord(raw rawLine) Record { return record } +// userTextForClassify returns the leading text of a user-role content +// value the way the parser sees it: the first text block, or the raw +// string content. +func userTextForClassify(content any) string { + switch v := content.(type) { + case string: + return v + case []any: + for _, item := range v { + block, ok := item.(map[string]any) + if !ok { + continue + } + if stringField(block, "type") == "text" { + return stringField(block, "text") + } + return "" + } + } + return "" +} + func normalizeCodexRecord(raw rawLine) Record { payload := decodeObject(raw.Payload) payloadType := stringField(payload, "type") @@ -595,6 +692,20 @@ func normalizeCodexRecord(raw rawLine) Record { record.Text = joinNonEmpty(" ", "turn", record.TurnID, stringField(payload, "model"), record.Workspace) case "event_msg": record.Type = joinNonEmpty(":", raw.Type, payloadType) + if payloadType == "item_completed" { + // Codex 0.147: the visible transcript is item_completed + // UserMessage/AgentMessage TurnItems (docs/design/0004). + if message, ok := codex.DecodeCompletedTurnMessage(raw.Payload); ok { + record.UUID = message.ID + record.Role = message.Role + record.Kind = "assistant_message" + if message.Role == "user" { + record.Kind = "user_prompt" + } + record.Text = truncateText(cleanText(message.Text), 1000) + break + } + } normalizeCodexEvent(&record, payload, payloadType) case "response_item": record.Type = joinNonEmpty(":", raw.Type, payloadType) @@ -694,6 +805,42 @@ func normalizeCodexResponseItem(record *Record, payload map[string]any, payloadT } } +// demoteLegacyCodexMessages applies the parser's one-source-per-rollout +// rule (docs/design/0004) to a rollout's records: when the file carries +// completed TurnItem messages, the legacy event_msg user_message / +// agent_message events and the raw response_item messages are model +// I/O or duplicates, not the conversation. They stay in the log as +// records — nothing is dropped — but no longer count as prompts or +// replies, so a 0.147 rollout is not double-counted and an injected +// instruction envelope is not shown as something the human typed. +func demoteLegacyCodexMessages(records []Record) bool { + hasCompleted := false + for i := range records { + if records[i].Type == "event_msg:item_completed" && (records[i].Kind == "user_prompt" || records[i].Kind == "assistant_message") { + hasCompleted = true + break + } + } + if !hasCompleted { + return false + } + for i := range records { + r := &records[i] + switch r.Type { + case "event_msg:user_message", "event_msg:agent_message": + r.Kind = "legacy_message" + case "response_item:message": + switch r.Kind { + case "user_prompt": + r.Kind = "model_input" + case "assistant_message": + r.Kind = "model_output" + } + } + } + return true +} + func relationFor(sessionStart, sessionEnd, scopeStart, scopeEnd time.Time) ScopeRelation { return ScopeRelation{ OverlapsScope: sessionStart.Before(scopeEnd) && !sessionEnd.Before(scopeStart), diff --git a/internal/sessionlog/sessionlog_test.go b/internal/sessionlog/sessionlog_test.go index e52de16..b72cd46 100644 --- a/internal/sessionlog/sessionlog_test.go +++ b/internal/sessionlog/sessionlog_test.go @@ -3,6 +3,7 @@ package sessionlog import ( "os" "path/filepath" + "strings" "testing" "time" ) @@ -151,6 +152,151 @@ func TestCollectClassifiesCodexDeveloperMessageAsInstruction(t *testing.T) { } } +// Codex 0.147 rollouts carry the visible conversation as +// event_msg.item_completed UserMessage/AgentMessage TurnItems; the raw +// response_item messages are model I/O (with injected envelopes) and +// legacy user_message/agent_message events are duplicates in hybrid +// files. The log surface must apply the parser's one-source rule: +// item_completed rows become the prompts/replies with text, and the +// duplicates stay as records but stop counting (docs/design/0004). +func TestCollectCodex0147ItemCompletedIsTheConversation(t *testing.T) { + root := t.TempDir() + home := filepath.Join(root, ".codex") + sessionDir := filepath.Join(home, "sessions", "2026", "08") + if err := os.MkdirAll(sessionDir, 0755); err != nil { + t.Fatal(err) + } + file := filepath.Join(sessionDir, "rollout-0147.jsonl") + writeLines(t, file, + `{"timestamp":"2026-08-18T04:45:00Z","type":"session_meta","payload":{"id":"codex-0147","cwd":"/tmp/repo"}}`, + `{"timestamp":"2026-08-18T04:45:01Z","type":"response_item","payload":{"type":"message","role":"user","content":[{"type":"input_text","text":"# AGENTS.md instructions for /tmp/repo (injected envelope)"}]}}`, + `{"timestamp":"2026-08-18T04:45:02Z","type":"event_msg","payload":{"type":"item_completed","item":{"id":"item-u1","type":"UserMessage","content":[{"type":"text","text":"hello from 0.147"}]}}}`, + `{"timestamp":"2026-08-18T04:45:03Z","type":"event_msg","payload":{"type":"user_message","message":"hello from 0.147"}}`, + `{"timestamp":"2026-08-18T04:45:04Z","type":"event_msg","payload":{"type":"item_completed","item":{"id":"item-a1","type":"AgentMessage","content":[{"type":"Text","text":"Received."}]}}}`, + `{"timestamp":"2026-08-18T04:45:05Z","type":"response_item","payload":{"type":"message","role":"assistant","content":[{"type":"output_text","text":"Received."}]}}`, + `{"timestamp":"2026-08-18T04:45:06Z","type":"event_msg","payload":{"type":"item_completed","item":{"id":"item-c1","type":"CommandExecution","command":"ls"}}}`, + ) + + start := mustParseTime(t, "2026-08-18T00:00:00Z") + end := mustParseTime(t, "2026-08-19T00:00:00Z") + bundle, err := Collect([]Source{{Provider: "codex", Home: home}}, Options{Start: start, End: end}) + if err != nil { + t.Fatal(err) + } + if len(bundle.Records) != 7 { + t.Fatalf("records = %d, want 7 (nothing dropped)", len(bundle.Records)) + } + byLine := map[int]Record{} + for _, r := range bundle.Records { + byLine[r.Line] = r + } + if r := byLine[3]; r.Kind != "user_prompt" || r.Role != "user" || r.Text != "hello from 0.147" || r.UUID != "item-u1" { + t.Fatalf("item_completed UserMessage: %+v", r) + } + if r := byLine[5]; r.Kind != "assistant_message" || r.Text != "Received." || r.UUID != "item-a1" { + t.Fatalf("item_completed AgentMessage: %+v", r) + } + if r := byLine[2]; r.Kind != "model_input" { + t.Fatalf("raw response_item user message must be model_input, got %+v", r) + } + if r := byLine[4]; r.Kind != "legacy_message" { + t.Fatalf("legacy user_message event must be demoted in a hybrid file, got %+v", r) + } + if r := byLine[6]; r.Kind != "model_output" { + t.Fatalf("raw response_item assistant message must be model_output, got %+v", r) + } + if r := byLine[7]; r.Kind != "item_completed" { + t.Fatalf("non-message TurnItem keeps its own kind, got %+v", r) + } + if bundle.Metrics.UserPrompts != 1 || bundle.Metrics.AssistantMessages != 1 { + t.Fatalf("metrics must count the conversation once: %+v", bundle.Metrics) + } + if bundle.Sessions[0].Preview != "hello from 0.147" { + t.Fatalf("preview must come from the visible prompt, got %q", bundle.Sessions[0].Preview) + } +} + +// A legacy (pre-0.147) rollout has no item_completed messages, so its +// user_message / agent_message events remain the conversation. +func TestCollectCodexLegacyRolloutUnchanged(t *testing.T) { + root := t.TempDir() + home := filepath.Join(root, ".codex") + sessionDir := filepath.Join(home, "sessions", "2026", "05") + if err := os.MkdirAll(sessionDir, 0755); err != nil { + t.Fatal(err) + } + writeLines(t, filepath.Join(sessionDir, "legacy.jsonl"), + `{"timestamp":"2026-05-21T00:01:00Z","type":"session_meta","payload":{"id":"codex-legacy","cwd":"/tmp/repo"}}`, + `{"timestamp":"2026-05-21T00:02:00Z","type":"event_msg","payload":{"type":"user_message","message":"legacy prompt"}}`, + `{"timestamp":"2026-05-21T00:03:00Z","type":"event_msg","payload":{"type":"agent_message","message":"legacy reply"}}`, + ) + bundle, err := Collect([]Source{{Provider: "codex", Home: home}}, Options{Start: mustParseTime(t, "2026-05-21T00:00:00Z"), End: mustParseTime(t, "2026-05-22T00:00:00Z")}) + if err != nil { + t.Fatal(err) + } + if bundle.Metrics.UserPrompts != 1 || bundle.Metrics.AssistantMessages != 1 { + t.Fatalf("legacy rollout must keep counting: %+v", bundle.Metrics) + } +} + +// user-role lines that no human typed — command markers, local command +// echoes, task notifications, injected meta (skill bodies), compaction +// carriers — must not be user_prompt in the log, exactly as in the +// parser; --kind user_prompt is "the humans in the loop". +func TestCollectClaudeUserRoleNoiseIsNotAPrompt(t *testing.T) { + root := t.TempDir() + home := filepath.Join(root, ".claude") + projDir := filepath.Join(home, "projects", "-tmp-repo") + if err := os.MkdirAll(projDir, 0755); err != nil { + t.Fatal(err) + } + writeLines(t, filepath.Join(projDir, "s1.jsonl"), + `{"type":"user","uuid":"u1","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:00Z","message":{"role":"user","content":[{"type":"text","text":"real prompt"}]}}`, + `{"type":"user","uuid":"u2","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:01Z","message":{"role":"user","content":"/modelmodel"}}`, + `{"type":"user","uuid":"u3","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:02Z","message":{"role":"user","content":[{"type":"text","text":"Set model"}]}}`, + `{"type":"user","uuid":"u4","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:03Z","message":{"role":"user","content":[{"type":"text","text":"done"}]}}`, + `{"type":"user","uuid":"u5","sessionId":"s1","cwd":"/tmp/repo","isMeta":true,"timestamp":"2026-08-18T10:00:04Z","message":{"role":"user","content":[{"type":"text","text":"Base directory for this skill: /x"}]}}`, + `{"type":"user","uuid":"u6","sessionId":"s1","cwd":"/tmp/repo","isCompactSummary":true,"timestamp":"2026-08-18T10:00:05Z","message":{"role":"user","content":[{"type":"text","text":"summary of earlier context"}]}}`, + `{"type":"user","uuid":"u7","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:06Z","message":{"role":"user","content":[{"type":"tool_result","content":"ok"}]}}`, + ) + bundle, err := Collect([]Source{{Provider: "claude-code", Home: home}}, Options{Start: mustParseTime(t, "2026-08-18T00:00:00Z"), End: mustParseTime(t, "2026-08-19T00:00:00Z")}) + if err != nil { + t.Fatal(err) + } + want := map[string]string{"u1": "user_prompt", "u2": "command", "u3": "command_output", "u4": "notification", "u5": "meta", "u6": "compact_summary", "u7": "tool_result"} + for _, r := range bundle.Records { + if want[r.UUID] != r.Kind { + t.Errorf("%s: kind %q, want %q", r.UUID, r.Kind, want[r.UUID]) + } + } + if bundle.Metrics.UserPrompts != 1 { + t.Fatalf("user prompts: %+v", bundle.Metrics) + } + + // --kind and --match narrow the records list, keep scope-wide + // metrics, and report the matched count; Match sees the raw line. + bundle, err = Collect([]Source{{Provider: "claude-code", Home: home}}, Options{ + Start: mustParseTime(t, "2026-08-18T00:00:00Z"), End: mustParseTime(t, "2026-08-19T00:00:00Z"), + Kinds: []string{"user_prompt", "meta"}, + }) + if err != nil { + t.Fatal(err) + } + if len(bundle.Records) != 2 || bundle.Metrics.Records != 7 || bundle.Metrics.RecordsMatched != 2 || bundle.Metrics.RecordsReturned != 2 { + t.Fatalf("kind filter: %d records, metrics %+v", len(bundle.Records), bundle.Metrics) + } + bundle, err = Collect([]Source{{Provider: "claude-code", Home: home}}, Options{ + Start: mustParseTime(t, "2026-08-18T00:00:00Z"), End: mustParseTime(t, "2026-08-19T00:00:00Z"), + Match: func(line string) bool { return strings.Contains(line, "isMeta") }, + }) + if err != nil { + t.Fatal(err) + } + if len(bundle.Records) != 1 || bundle.Records[0].UUID != "u5" { + t.Fatalf("match filter on raw line: %+v", bundle.Records) + } +} + func writeLines(t *testing.T, path string, lines ...string) { t.Helper() content := "" diff --git a/skills/ccx/SKILL.md b/skills/ccx/SKILL.md index 1932e7b..9864ac5 100644 --- a/skills/ccx/SKILL.md +++ b/skills/ccx/SKILL.md @@ -44,6 +44,8 @@ ccx │ └── --since / --until TIME # RFC3339 or YYYY-MM-DD │ └── --tz ZONE # IANA timezone, UTC, local, or offset like +8 │ └── --json --raw # Evidence bundle, optional raw JSONL +│ └── --kind K1,K2 # Only these record kinds (user_prompt, assistant_message, tool_call, ...) +│ └── --match PHRASE [-w] # Only records whose raw line contains the phrase ├── insight [project] # HTML/JSON data report from session logs │ └── --scope --tz --since --until --all │ └── --json # Aggregates: days[]/providers[]/workspaces[] @@ -72,6 +74,8 @@ ccx view abc123 # View by session ID (prefix match) ccx export -f html # Export to HTML ccx sessions --scope yesterday --tz +8 --all --json # Session containers by end time ccx log --scope yesterday --tz +8 --all --json +ccx log --scope today --all --kind user_prompt # The humans in the loop: every prompt today, all sessions, in order +ccx log --scope month --all --match deadman -w # When a term came up inside a window (raw-line match) ccx trace # Outline of the latest workspace session ccx trace abc123 --turn 5 # Full evidence for one turn ccx related abc123 # Sessions connected to this one: fork, handoff, mentions, shared files From 3a8644f06a3e1bba39b073a0f6f43ce5986224e8 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 06:00:05 -0700 Subject: [PATCH 05/13] =?UTF-8?q?feat(trace,log):=20human=20interventions?= =?UTF-8?q?=20=E2=80=94=20interrupts=20and=20denials;=20label=20tool-only?= =?UTF-8?q?=20steps?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The human in the loop leaves two marks that ccx misread. The "[Request interrupted by user]" marker (the human pressed stop) is a plain user-role text line, so it classified as a prompt and opened a fake turn — `u: [Request interrupted by user for tool use]`. A permission-prompt rejection ("The user doesn't want to proceed with this tool use") is a tool_result with is_error, so it counted as a tool error. One real store holds 650 interruptions across 321 sessions and 145 rejections; none were visible as what they are. - parser: KindInterrupt (harness marker via ClassifyUserText; never an exchange anchor, so no fake turn) and IsToolDenial. - trace: Turn/Step/TraceStats gain interrupts and denials; the rejected call's evidence is marked denied (not an error — it did not run); outline header ("1 interrupt, 2 denied"), turn badges, step badges ("[4t 1! 1d]"), OutlineTurn/OutlineStep fields. - trace: a narration-less step gets a headline from its tools — "(no narration) Bash x3, Read" — instead of a bare badge row (2026-08-17 dogfood finding 3). - log: kinds `interrupt` and `tool_denied`; Claude tool_result rows now preview their content instead of the literal word tool_result. Tests: TestClassifyUserTextHarnessWrappers (+interrupt cases), TestIsToolDenial, TestAnalyzeCountsInterruptsAndDenials, TestOutlineLabelsToolOnlySteps, sessionlog kinds test extended. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 5 ++ docs/schema.md | 8 +++ internal/parser/classify_test.go | 20 ++++++ internal/parser/session.go | 13 ++++ internal/parser/types.go | 1 + internal/sessionlog/sessionlog.go | 11 +++ internal/sessionlog/sessionlog_test.go | 6 +- internal/trace/analysis.go | 59 ++++++++++++++++ internal/trace/analysis_test.go | 97 ++++++++++++++++++++++++++ internal/trace/outline.go | 91 ++++++++++++++++++++++-- internal/trace/types.go | 30 ++++++-- skills/ccx/SKILL.md | 4 +- 12 files changed, 330 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e92db3a..e33d178 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,12 +13,17 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) - **`ccx search --hits` turns matches into citations.** One row per matching message — time, session, role, message id, quote — oldest first across sessions, `-n`-capped with a visible "showing N of M". The anchors are the same ones `trace` and `view` use, so a claim built on a search can point at its evidence (design: docs/design/0005-evidence-citations-lessons-from-semantica.md). Under `--raw` the unit is a transcript line, anchored by its own `uuid`/`type`/`timestamp`. - **`ccx search --content` reports when: `FIRST` column, `first_hit` in `--json`, `--sort first|last|hits`.** "When did we first mention X" needs the earliest matching message and an oldest-first order; results only carried session end time and ranked by hit count. Each content hit now records the timestamp of its earliest matching message (parsed messages by default; the raw line's top-level `timestamp` under `--raw`), printed as `FIRST` and sortable with `--sort first`; `--sort last` orders by session activity; `--sort hits` is the old order and the default. +### Added +- **Human interventions are first-class in `trace` and `log`.** "[Request interrupted by user]" (the human pressed stop) and permission-prompt rejections ("The user doesn't want to proceed with this tool use") are the human in the loop, but ccx read the first as a *prompt* — it opened a fake turn `u: [Request interrupted by user for tool use]` — and the second as an ordinary tool error. New parser kind `interrupt` (harness marker, never an exchange anchor) and `parser.IsToolDenial`; `trace` counts `interrupts` and `denials` on turn, step, and stats, marks the rejected call `denied` (not an error, it did not run), and badges them in the outline header and rows (`1 interrupt, 2 denied`; step `[4t 1! 1d]`); `log` reports kinds `interrupt` and `tool_denied` (`ccx log --scope today --all --kind interrupt,tool_denied`). 650 interruptions across 321 sessions and 145 rejections in one real store were invisible before. + ### Changed +- **`ccx trace` labels narration-less steps.** A step the agent never narrated (straight to tools) rendered as a bare badge row `1. [4t]`; the outline now says what it ran: `(no narration) Bash x3, Read` (docs/devlog/2026-08-17-codex-0147-rollout-drift.org finding 3). - **`ccx log --kind` and `--match PHRASE [-w]` turn the firehose into a timeline.** `log` emitted every record in scope (16k for one day) with no way to narrow; "what did the humans ask today, across every session" was not answerable. `--kind user_prompt,assistant_message` keeps only those kinds; `--match` keeps records whose raw transcript line contains the phrase (grep parity, `-w` for whole words) — the time-bounded complement of `search --hits`. Metrics stay honest: `records` is scope-wide, new `records_matched` is the narrowed count, `showing` is after `-n`. - **`ccx search --content` is ~10x faster and shows progress.** The scan ran on one core and lowercased every transcript line: 6m18s cold / 1m14s warm over a 3.5 GB store, silent throughout. Sessions now scan on a bounded worker pool (up to 8), the raw prefilter matches case-insensitively without allocating and stops at the first hit, and a `scanning N/M sessions` line ticks on stderr when it is a terminal. Same store, warm: 7.6s. ### Fixed - **`ccx log` now applies the same conversation rules as the parser.** Two classes of user-role lines were reported as `user_prompt`: Claude harness wrappers (slash-command markers, `` echoes, task notifications), injected meta messages (skill bodies), and compaction carriers — 233 "prompts" today of which 79 were typed by a human; and Codex 0.147 raw `response_item` messages, including the injected AGENTS.md envelope, while the real `item_completed` UserMessage/AgentMessage rows rendered as bare `item_completed` with no text. `sessionlog` now reuses `parser.ClassifyUserText` and the Codex TurnItem decoder (`DecodeCompletedTurnMessage`, exported), demotes legacy/raw duplicates in a 0.147 rollout to `legacy_message`/`model_input`/`model_output` (records kept, counts fixed), and re-tallies session kinds and preview after demotion. `user_prompts`/`assistant_messages` metrics and `insight` reports built on them count the conversation once. +- **`ccx log` previews Claude tool results.** A `tool_result` block's payload lives under `content`; the preview only looked at `text`/`message`, so every Claude tool result row read as the literal word `tool_result`. - **Heredoc bodies no longer count as shell redirects.** `extractRedirectPaths` scanned the whole Bash command, so a Go `if n > 0` or a markdown `> 2026-08-18` inside `python3 - <<'EOF'` / `cat > f <<'EOF'` became "edited files" (`.../0`, `.../2026-08-18`) in `trace` `files_edited` and in session connections. Heredoc bodies are stripped before the redirect scan. - **`-n` is the `--limit` shorthand everywhere.** Only `search` had it; `sessions -n 2` failed with "unknown shorthand flag". `sessions`, `projects`, and `log` now accept `-n` too. - **A session whose summary matched dropped its content evidence.** The summary hit short-circuited the content scan, so under `--content` the session where the term was actually discussed could be the one result with no hit count, previews, or first-hit time. Summary hits stay typed `session` but now carry the content fields. diff --git a/docs/schema.md b/docs/schema.md index bedf37c..41776e1 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -50,6 +50,14 @@ gaps, each capped at 5 minutes. Continuous work counts fully; an overnight gap counts as at most one cap. Use active time for "how long did this actually take"; use wall-span for calendar placement. +### Human interventions + +`interrupts` (the "[Request interrupted by user]" marker: the human +pressed stop) and `denials` (a tool call rejected at the permission +prompt) are counted on turns, steps, and `stats`; a denied call carries +`denied: true` on its evidence and is not counted as an error (it did +not run). Neither is a prompt: an interrupt never opens a turn. + ### Facts, not judgment Traces record what happened: text excerpts, tool calls, mutations, diff --git a/internal/parser/classify_test.go b/internal/parser/classify_test.go index 27791bd..0b7eec2 100644 --- a/internal/parser/classify_test.go +++ b/internal/parser/classify_test.go @@ -14,6 +14,8 @@ func TestClassifyUserTextHarnessWrappers(t *testing.T) { {"Caveat: generated by local commands", KindCommandOutput, true}, {"\nabc", KindNotification, true}, {" leading whitespace", KindNotification, true}, + {"[Request interrupted by user]", KindInterrupt, true}, + {"[Request interrupted by user for tool use]", KindInterrupt, true}, {"fix the login bug", KindUnknown, false}, {"", KindUnknown, false}, } @@ -68,3 +70,21 @@ func TestComputeStatsExcludesHarnessTurns(t *testing.T) { t.Errorf("UserPrompts = %d, want 1", stats.UserPrompts) } } + +func TestIsToolDenial(t *testing.T) { + yes := []string{ + "The user doesn't want to proceed with this tool use. The tool use was rejected (eg. if it was a file edit, the new_string was NOT written to the file).", + " The user doesn't want to take this action right now. STOP what you are doing", + } + no := []string{"Exit code 137\n[Request interrupted by user for tool use]", "permission denied: /etc/shadow", ""} + for _, s := range yes { + if !IsToolDenial(s) { + t.Errorf("IsToolDenial(%q) = false, want true", s) + } + } + for _, s := range no { + if IsToolDenial(s) { + t.Errorf("IsToolDenial(%q) = true, want false", s) + } + } +} diff --git a/internal/parser/session.go b/internal/parser/session.go index 4a99cd2..2857a44 100644 --- a/internal/parser/session.go +++ b/internal/parser/session.go @@ -294,10 +294,23 @@ func ClassifyUserText(text string) (MessageKind, bool) { return KindCommandOutput, true case strings.HasPrefix(t, ""): return KindNotification, true + case strings.HasPrefix(t, "[Request interrupted by user"): + // The human pressed stop: a harness marker, not a prompt. It + // belongs to the turn it interrupted (no new exchange). + return KindInterrupt, true } return KindUnknown, false } +// IsToolDenial reports whether a tool result is the harness telling +// the agent that the human rejected the call at the permission prompt +// (Claude Code) — a human intervention, distinct from a tool error. +func IsToolDenial(text string) bool { + t := strings.TrimSpace(text) + return strings.HasPrefix(t, "The user doesn't want to proceed with this tool use") || + strings.HasPrefix(t, "The user doesn't want to take this action right now") +} + // extractCommandName extracts command name from /foo func extractCommandName(text string) string { start := strings.Index(text, "") diff --git a/internal/parser/types.go b/internal/parser/types.go index 0e57882..97f57dc 100644 --- a/internal/parser/types.go +++ b/internal/parser/types.go @@ -14,6 +14,7 @@ const ( KindCommand MessageKind = "command" // Slash command (/init, /compact, etc) KindCommandOutput MessageKind = "command_output" // harness echo KindNotification MessageKind = "notification" // background-task event + KindInterrupt MessageKind = "interrupt" // [Request interrupted by user...] — the human stopped the agent KindMeta MessageKind = "meta" // Meta/system instruction KindCompactSummary MessageKind = "compact_summary" // Compacted context carrier KindAssistant MessageKind = "assistant" // Assistant response diff --git a/internal/sessionlog/sessionlog.go b/internal/sessionlog/sessionlog.go index eaf8b41..ec43fb4 100644 --- a/internal/sessionlog/sessionlog.go +++ b/internal/sessionlog/sessionlog.go @@ -623,6 +623,9 @@ func normalizeClaudeRecord(raw rawLine) Record { record.Kind = "meta" case contentHasType(content, "tool_result"): record.Kind = "tool_result" + if parser.IsToolDenial(record.Text) { + record.Kind = "tool_denied" + } default: if kind, ok := parser.ClassifyUserText(userTextForClassify(content)); ok { record.Kind = string(kind) @@ -1012,6 +1015,14 @@ func contentPreview(value any) string { return s } } + // tool_result blocks carry their payload under "content" (a + // string or nested blocks); without this a Claude tool result + // previewed as the literal word "tool_result". + if nested, ok := v["content"]; ok { + if s := contentPreview(nested); s != "" { + return s + } + } if t := stringField(v, "type"); t != "" { return t } diff --git a/internal/sessionlog/sessionlog_test.go b/internal/sessionlog/sessionlog_test.go index b72cd46..22e50da 100644 --- a/internal/sessionlog/sessionlog_test.go +++ b/internal/sessionlog/sessionlog_test.go @@ -258,12 +258,14 @@ func TestCollectClaudeUserRoleNoiseIsNotAPrompt(t *testing.T) { `{"type":"user","uuid":"u5","sessionId":"s1","cwd":"/tmp/repo","isMeta":true,"timestamp":"2026-08-18T10:00:04Z","message":{"role":"user","content":[{"type":"text","text":"Base directory for this skill: /x"}]}}`, `{"type":"user","uuid":"u6","sessionId":"s1","cwd":"/tmp/repo","isCompactSummary":true,"timestamp":"2026-08-18T10:00:05Z","message":{"role":"user","content":[{"type":"text","text":"summary of earlier context"}]}}`, `{"type":"user","uuid":"u7","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:06Z","message":{"role":"user","content":[{"type":"tool_result","content":"ok"}]}}`, + `{"type":"user","uuid":"u8","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:07Z","message":{"role":"user","content":[{"type":"text","text":"[Request interrupted by user]"}]}}`, + `{"type":"user","uuid":"u9","sessionId":"s1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:08Z","message":{"role":"user","content":[{"type":"tool_result","content":"The user doesn't want to proceed with this tool use. The tool use was rejected.","is_error":true}]}}`, ) bundle, err := Collect([]Source{{Provider: "claude-code", Home: home}}, Options{Start: mustParseTime(t, "2026-08-18T00:00:00Z"), End: mustParseTime(t, "2026-08-19T00:00:00Z")}) if err != nil { t.Fatal(err) } - want := map[string]string{"u1": "user_prompt", "u2": "command", "u3": "command_output", "u4": "notification", "u5": "meta", "u6": "compact_summary", "u7": "tool_result"} + want := map[string]string{"u1": "user_prompt", "u2": "command", "u3": "command_output", "u4": "notification", "u5": "meta", "u6": "compact_summary", "u7": "tool_result", "u8": "interrupt", "u9": "tool_denied"} for _, r := range bundle.Records { if want[r.UUID] != r.Kind { t.Errorf("%s: kind %q, want %q", r.UUID, r.Kind, want[r.UUID]) @@ -282,7 +284,7 @@ func TestCollectClaudeUserRoleNoiseIsNotAPrompt(t *testing.T) { if err != nil { t.Fatal(err) } - if len(bundle.Records) != 2 || bundle.Metrics.Records != 7 || bundle.Metrics.RecordsMatched != 2 || bundle.Metrics.RecordsReturned != 2 { + if len(bundle.Records) != 2 || bundle.Metrics.Records != 9 || bundle.Metrics.RecordsMatched != 2 || bundle.Metrics.RecordsReturned != 2 { t.Fatalf("kind filter: %d records, metrics %+v", len(bundle.Records), bundle.Metrics) } bundle, err = Collect([]Source{{Provider: "claude-code", Home: home}}, Options{ diff --git a/internal/trace/analysis.go b/internal/trace/analysis.go index 3e793de..0206c39 100644 --- a/internal/trace/analysis.go +++ b/internal/trace/analysis.go @@ -66,6 +66,8 @@ func Analyze(session *parser.Session) *TraceResult { allTools := make(map[string]struct{}) stepCount := 0 toolErrors := 0 + interrupts := 0 + denials := 0 var mainCost float64 var activeSecs float64 var inputTok, outputTok, cacheReadTok, cacheCreateTok, reasoningTok int @@ -82,6 +84,8 @@ func Analyze(session *parser.Session) *TraceResult { } stepCount += len(turn.Steps) toolErrors += turn.Errors + interrupts += turn.Interrupts + denials += turn.Denials mainCost += turn.CostUSD activeSecs += turn.ActiveSecs inputTok += turn.InputTokens @@ -112,6 +116,8 @@ func Analyze(session *parser.Session) *TraceResult { FilesRead: len(allRead), ToolsUsed: len(allTools), ToolErrors: toolErrors, + Interrupts: interrupts, + Denials: denials, InputTokens: inputTok, OutputTokens: outputTok, CacheReadTokens: cacheReadTok, @@ -245,6 +251,13 @@ func buildTurn(index int, anchor *parser.Message, messages []*parser.Message, si turn.CostUSD += msg.Usage.CostUSD } + if msg.Kind == parser.KindInterrupt { + turn.Interrupts++ + if len(steps) > 0 { + steps[len(steps)-1].Interrupts++ + } + } + if msg.Kind == parser.KindAssistant { if narration := firstText(msg); narration != "" { step := Step{ @@ -337,6 +350,28 @@ func buildTurn(index int, anchor *parser.Message, messages []*parser.Message, si mutIdxByToolID[cb.ToolID] = [2]int{len(steps) - 1, len(step.Mutations) - 1} } case "tool_result": + if denied := parser.IsToolDenial(toolResultText(cb.ToolResult)); denied { + // A human rejection is an intervention, not an + // error: count it on the turn/step and mark the + // issuing call as denied (materialized like errors + // so denied lists match the counts). + turn.Denials++ + if step := stepForResult(msg, steps, stepByToolID); step != nil { + step.Denials++ + } + if cb.ToolID != "" { + if loc, ok := mutIdxByToolID[cb.ToolID]; ok { + steps[loc[0]].Mutations[loc[1]].Denied = true + } else if ev, ok := callByToolID[cb.ToolID]; ok { + ev.Denied = true + if idx, ok := stepByToolID[cb.ToolID]; ok && idx >= 0 && idx < len(steps) { + steps[idx].Mutations = append(steps[idx].Mutations, ev) + mutIdxByToolID[cb.ToolID] = [2]int{idx, len(steps[idx].Mutations) - 1} + } + } + } + continue + } if !cb.IsError { continue } @@ -372,6 +407,30 @@ func buildTurn(index int, anchor *parser.Message, messages []*parser.Message, si return turn } +// toolResultText returns the text of a tool result payload, which +// arrives as a string or as a list of content blocks. +func toolResultText(result any) string { + switch v := result.(type) { + case string: + return v + case []any: + var parts []string + for _, item := range v { + if block, ok := item.(map[string]any); ok { + if text, ok := block["text"].(string); ok { + parts = append(parts, text) + } + } + } + return strings.Join(parts, "\n") + case map[string]any: + if text, ok := v["text"].(string); ok { + return text + } + } + return "" +} + // stepForResult finds the step that issued the call a result belongs // to (via ToolID), falling back to the latest step. Results can land // after later narration — background agents especially — so ToolID diff --git a/internal/trace/analysis_test.go b/internal/trace/analysis_test.go index 4200a1f..b202d96 100644 --- a/internal/trace/analysis_test.go +++ b/internal/trace/analysis_test.go @@ -570,3 +570,100 @@ func TestExtractRedirectPathsIgnoresHeredocBodies(t *testing.T) { t.Fatalf("unterminated heredoc: got %v", got) } } + +// Human interventions inside a turn: the "[Request interrupted by +// user]" marker and a permission-prompt rejection are counted on the +// turn, the step, and the stats; the denied call is marked and is not +// an error; the interrupt does not open a new turn (it once rendered +// as a fake "u: [Request interrupted by user for tool use]" turn). +func TestAnalyzeCountsInterruptsAndDenials(t *testing.T) { + now := time.Now() + session := &parser.Session{ + ID: "human", StartTime: now, EndTime: now.Add(10 * time.Minute), + RootMessages: []*parser.Message{ + {UUID: "u1", Kind: parser.KindUserPrompt, Type: "user", Timestamp: now, + Content: []parser.ContentBlock{{Type: "text", Text: "clean up the disk"}}}, + {UUID: "a1", Kind: parser.KindAssistant, Type: "assistant", Timestamp: now.Add(time.Minute), + Content: []parser.ContentBlock{ + {Type: "text", Text: "Deleting the caches."}, + {Type: "tool_use", ToolName: "Bash", ToolID: "t1", ToolInput: map[string]any{"command": "rm -rf ~/Library/Caches"}}, + }}, + {UUID: "r1", Kind: parser.KindToolResult, Type: "user", Timestamp: now.Add(2 * time.Minute), + Content: []parser.ContentBlock{{Type: "tool_result", ToolID: "t1", IsError: true, + ToolResult: "The user doesn't want to proceed with this tool use. The tool use was rejected."}}}, + {UUID: "a2", Kind: parser.KindAssistant, Type: "assistant", Timestamp: now.Add(3 * time.Minute), + Content: []parser.ContentBlock{ + {Type: "text", Text: "Understood, listing instead."}, + {Type: "tool_use", ToolName: "Bash", ToolID: "t2", ToolInput: map[string]any{"command": "du -sh ~/Library/*"}}, + }}, + {UUID: "r2", Kind: parser.KindToolResult, Type: "user", Timestamp: now.Add(4 * time.Minute), + Content: []parser.ContentBlock{{Type: "tool_result", ToolID: "t2", IsError: true, + ToolResult: []any{map[string]any{"type": "text", "text": "Exit code 137\n[Request interrupted by user for tool use]"}}}}}, + {UUID: "i1", Kind: parser.KindInterrupt, Type: "user", Timestamp: now.Add(4*time.Minute + time.Second), + Content: []parser.ContentBlock{{Type: "text", Text: "[Request interrupted by user for tool use]"}}}, + {UUID: "u2", Kind: parser.KindUserPrompt, Type: "user", Timestamp: now.Add(5 * time.Minute), + Content: []parser.ContentBlock{{Type: "text", Text: "just report sizes, do not delete"}}}, + }, + } + result := Analyze(session) + if result.Stats.TurnCount != 2 { + t.Fatalf("turns: got %d, want 2 (interrupt must not open a turn)", result.Stats.TurnCount) + } + turn := result.Turns[0] + if turn.Interrupts != 1 || turn.Denials != 1 { + t.Fatalf("turn interventions: interrupts=%d denials=%d", turn.Interrupts, turn.Denials) + } + // The denial is not an error; the killed command (exit 137) is. + if turn.Errors != 1 { + t.Fatalf("errors: got %d, want 1", turn.Errors) + } + if len(turn.Steps) != 2 || turn.Steps[0].Denials != 1 || turn.Steps[1].Interrupts != 1 { + t.Fatalf("step attribution: %+v", turn.Steps) + } + var denied *ToolCallEvidence + for i := range turn.Steps[0].Mutations { + if turn.Steps[0].Mutations[i].ToolID == "t1" { + denied = &turn.Steps[0].Mutations[i] + } + } + if denied == nil || !denied.Denied || denied.IsError { + t.Fatalf("denied call evidence: %+v", denied) + } + if result.Stats.Interrupts != 1 || result.Stats.Denials != 1 { + t.Fatalf("stats: %+v", result.Stats) + } + outline := BuildOutline(result, 80) + if outline.Turns[0].Interrupts != 1 || outline.Turns[0].Denials != 1 || outline.Turns[0].Steps[0].Denials != 1 { + t.Fatalf("outline: %+v", outline.Turns[0]) + } + text := RenderOutlineText(outline) + if !strings.Contains(text, "1 interrupt, 1 denied") || !strings.Contains(text, "1 denied)") { + t.Fatalf("outline text must badge interventions:\n%s", text) + } +} + +// A step with tools but no narration must still get a readable +// headline in the outline (it rendered as a bare "[4t]" badge row). +func TestOutlineLabelsToolOnlySteps(t *testing.T) { + now := time.Now() + session := &parser.Session{ + ID: "tools-only", StartTime: now, EndTime: now.Add(time.Minute), + RootMessages: []*parser.Message{ + {UUID: "u1", Kind: parser.KindUserPrompt, Type: "user", Timestamp: now, + Content: []parser.ContentBlock{{Type: "text", Text: "try open chrome"}}}, + {UUID: "a1", Kind: parser.KindAssistant, Type: "assistant", Timestamp: now.Add(time.Second), + Content: []parser.ContentBlock{ + {Type: "tool_use", ToolName: "Bash", ToolID: "t1", ToolInput: map[string]any{"command": "open -a chrome"}}, + {Type: "tool_use", ToolName: "Bash", ToolID: "t2", ToolInput: map[string]any{"command": "which chrome"}}, + {Type: "tool_use", ToolName: "Read", ToolID: "t3", ToolInput: map[string]any{"file_path": "/w/x"}}, + {Type: "tool_use", ToolName: "Glob", ToolID: "t4", ToolInput: map[string]any{"pattern": "*"}}, + {Type: "tool_use", ToolName: "Grep", ToolID: "t5", ToolInput: map[string]any{"pattern": "x"}}, + }}, + }, + } + outline := BuildOutline(Analyze(session), 80) + got := outline.Turns[0].Steps[0].Headline + if got != "(no narration) Bash x2, Glob, Grep, +1 more" { + t.Fatalf("tool-only headline: %q", got) + } +} diff --git a/internal/trace/outline.go b/internal/trace/outline.go index 5c271fb..f37b355 100644 --- a/internal/trace/outline.go +++ b/internal/trace/outline.go @@ -3,6 +3,7 @@ package trace import ( "fmt" "path/filepath" + "sort" "strings" "time" ) @@ -39,6 +40,8 @@ func BuildOutline(result *TraceResult, width int) *Outline { SupersededByTurn: turn.SupersededByTurn, Edits: len(turn.FilesEdited), Errors: turn.Errors, + Interrupts: turn.Interrupts, + Denials: turn.Denials, InputTokens: turn.InputTokens, OutputTokens: turn.OutputTokens, CacheReadTokens: turn.CacheReadTokens, @@ -57,15 +60,22 @@ func BuildOutline(result *TraceResult, width int) *Outline { } for _, step := range turn.Steps { os := OutlineStep{ - Index: step.Index, - Headline: headline(step.Narration, width), - Edits: len(step.FilesEdited), - Errors: step.Errors, - Agents: len(step.Sidechains), + Index: step.Index, + Headline: headline(step.Narration, width), + Edits: len(step.FilesEdited), + Errors: step.Errors, + Interrupts: step.Interrupts, + Denials: step.Denials, + Agents: len(step.Sidechains), } for _, n := range step.ToolCounts { os.Tools += n } + if os.Headline == "" { + // A step the agent never narrated (straight to tools) + // still needs a readable line: say what it ran. + os.Headline = toolsHeadline(step) + } ot.Agents += len(step.Sidechains) ot.Steps = append(ot.Steps, os) } @@ -74,6 +84,42 @@ func BuildOutline(result *TraceResult, width int) *Outline { return outline } +// toolsHeadline labels a narration-less step from its tool sequence: +// "(no narration) Bash x3, Read" — busiest tools first, capped at +// three names, so the outline never shows a bare badge row. +func toolsHeadline(step Step) string { + if len(step.ToolCounts) == 0 { + return "(no narration)" + } + type tc struct { + name string + n int + } + var list []tc + for name, n := range step.ToolCounts { + list = append(list, tc{name, n}) + } + sort.Slice(list, func(i, j int) bool { + if list[i].n != list[j].n { + return list[i].n > list[j].n + } + return list[i].name < list[j].name + }) + var parts []string + for i, t := range list { + if i == 3 { + parts = append(parts, fmt.Sprintf("+%d more", len(list)-3)) + break + } + if t.n > 1 { + parts = append(parts, fmt.Sprintf("%s x%d", t.name, t.n)) + } else { + parts = append(parts, t.name) + } + } + return "(no narration) " + strings.Join(parts, ", ") +} + // headline reduces evidence text to its first meaningful line, capped // at width runes for outline display (<= 0 = uncapped). func headline(text string, width int) string { @@ -135,11 +181,25 @@ func RenderOutlineText(outline *Outline) string { // Wall-span misleads on long sessions; print active time next to it // and say which timezone the rendered times are in (JSON carries // them in UTC — silent localization breaks cross-referencing). - fmt.Fprintf(&b, "%s -> %s (times %s) | active %s | %s | %d steps | %d files edited | %d tool errors%s\n", + // Human interventions belong in the header: a session with 4 + // interrupts and 2 denials reads very differently from a clean run. + humanSeg := "" + if outline.Stats.Interrupts > 0 || outline.Stats.Denials > 0 { + var parts []string + if outline.Stats.Interrupts > 0 { + parts = append(parts, plural(outline.Stats.Interrupts, "interrupt")) + } + if outline.Stats.Denials > 0 { + parts = append(parts, fmt.Sprintf("%d denied", outline.Stats.Denials)) + } + humanSeg = " | " + strings.Join(parts, ", ") + } + fmt.Fprintf(&b, "%s -> %s (times %s) | active %s | %s | %d steps | %d files edited | %d tool errors%s%s\n", formatOutlineTime(s.Start), formatOutlineTime(s.End), outlineZone(s.Start), formatActive(outline.Stats.ActiveSecs), turnsSeg, outline.Stats.StepCount, outline.Stats.FilesEdited, outline.Stats.ToolErrors, + humanSeg, headerCost(outline.Stats.TotalCostUSD, outline.Stats.AgentsCostUSD)) if tok := tokenSplit(outline.Stats.InputTokens, outline.Stats.OutputTokens, outline.Stats.CacheReadTokens, outline.Stats.CacheCreateTokens, @@ -193,6 +253,12 @@ func turnBadges(turn OutlineTurn) string { if turn.Errors > 0 { parts = append(parts, fmt.Sprintf("%d errors", turn.Errors)) } + if turn.Interrupts > 0 { + parts = append(parts, plural(turn.Interrupts, "interrupt")) + } + if turn.Denials > 0 { + parts = append(parts, fmt.Sprintf("%d denied", turn.Denials)) + } if turn.Agents > 0 { parts = append(parts, fmt.Sprintf("%d agents", turn.Agents)) } @@ -261,6 +327,12 @@ func stepBadges(step OutlineStep) string { if step.Errors > 0 { parts = append(parts, fmt.Sprintf("%dx", step.Errors)) } + if step.Interrupts > 0 { + parts = append(parts, fmt.Sprintf("%d!", step.Interrupts)) + } + if step.Denials > 0 { + parts = append(parts, fmt.Sprintf("%dd", step.Denials)) + } if step.Agents > 0 { parts = append(parts, fmt.Sprintf("%da", step.Agents)) } @@ -270,6 +342,13 @@ func stepBadges(step OutlineStep) string { return " [" + strings.Join(parts, " ") + "]" } +func plural(n int, word string) string { + if n == 1 { + return fmt.Sprintf("1 %s", word) + } + return fmt.Sprintf("%d %ss", n, word) +} + func commitBadge(shas []string) string { if len(shas) == 0 { return "" diff --git a/internal/trace/types.go b/internal/trace/types.go index a4febc0..779af20 100644 --- a/internal/trace/types.go +++ b/internal/trace/types.go @@ -70,6 +70,13 @@ type Turn struct { FilesRead []string `json:"files_read,omitempty"` ToolCounts map[string]int `json:"tool_counts,omitempty"` Errors int `json:"errors,omitempty"` + // Human interventions inside the turn: Interrupts counts + // "[Request interrupted by user]" markers (the human pressed + // stop); Denials counts tool calls the human rejected at the + // permission prompt. Both are the human in the loop, distinct + // from a new prompt and from tool errors. + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` // Token split for the main loop. Cache tokens dominate real cost // (cache writes bill at 1.25x input, and one long turn can write // hundreds of thousands), so a cost without them is unauditable. @@ -111,6 +118,8 @@ type Step struct { // are summarized by ToolCounts and the turn-level file lists. Mutations []ToolCallEvidence `json:"mutations,omitempty"` Errors int `json:"errors,omitempty"` + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` Sidechains []Sidechain `json:"sidechains,omitempty"` CostUSD float64 `json:"cost_usd,omitempty"` } @@ -130,6 +139,9 @@ type ToolCallEvidence struct { MutatesWorkspace bool `json:"mutates_workspace"` Reads bool `json:"reads"` IsError bool `json:"is_error,omitempty"` + // Denied: the human rejected this call at the permission prompt. + // Such a call is not an error and did not run. + Denied bool `json:"denied,omitempty"` } type Sidechain struct { @@ -218,6 +230,8 @@ type TraceStats struct { FilesRead int `json:"files_read"` ToolsUsed int `json:"tools_used"` ToolErrors int `json:"tool_errors"` + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` WorkspaceDocs int `json:"workspace_docs"` KnowledgeEntries int `json:"knowledge_entries"` CommitsLinked int `json:"commits_linked"` @@ -271,6 +285,8 @@ type OutlineTurn struct { Edits int `json:"edits,omitempty"` Tools int `json:"tools,omitempty"` Errors int `json:"errors,omitempty"` + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` Agents int `json:"agents,omitempty"` InputTokens int `json:"input_tokens,omitempty"` OutputTokens int `json:"output_tokens,omitempty"` @@ -283,10 +299,12 @@ type OutlineTurn struct { } type OutlineStep struct { - Index int `json:"index"` - Headline string `json:"headline"` - Tools int `json:"tools,omitempty"` - Edits int `json:"edits,omitempty"` - Errors int `json:"errors,omitempty"` - Agents int `json:"agents,omitempty"` + Index int `json:"index"` + Headline string `json:"headline"` + Tools int `json:"tools,omitempty"` + Edits int `json:"edits,omitempty"` + Errors int `json:"errors,omitempty"` + Interrupts int `json:"interrupts,omitempty"` + Denials int `json:"denials,omitempty"` + Agents int `json:"agents,omitempty"` } diff --git a/skills/ccx/SKILL.md b/skills/ccx/SKILL.md index 9864ac5..afdec8d 100644 --- a/skills/ccx/SKILL.md +++ b/skills/ccx/SKILL.md @@ -75,6 +75,7 @@ ccx export -f html # Export to HTML ccx sessions --scope yesterday --tz +8 --all --json # Session containers by end time ccx log --scope yesterday --tz +8 --all --json ccx log --scope today --all --kind user_prompt # The humans in the loop: every prompt today, all sessions, in order +ccx log --scope today --all --kind interrupt,tool_denied # Where humans stopped or refused the agent ccx log --scope month --all --match deadman -w # When a term came up inside a window (raw-line match) ccx trace # Outline of the latest workspace session ccx trace abc123 --turn 5 # Full evidence for one turn @@ -86,7 +87,8 @@ ccx web # Start web UI at localhost:8080 `ccx trace` prints a terminal-readable outline: every turn (user intent) broken into steps (the agent's own narration), with tool, -edit, error, active-time, and cost rollups. Read the outline whole, +edit, error, interrupt/denial (human interventions), active-time, and +cost rollups. Read the outline whole, then drill: `--turn N` for one turn's full evidence, `--full` for the complete bundle. JSON kinds and field semantics are documented in `docs/schema.md`; header times are local (stated as `times UTC+X`), From 8d8f77a50c5d2fc31f6222ab9f98a02e2641562b Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 06:02:08 -0700 Subject: [PATCH 06/13] =?UTF-8?q?feat(view):=20--at=20MESSAGE=5FID=20[--co?= =?UTF-8?q?ntext=20N]=20=E2=80=94=20walk=20from=20a=20citation=20to=20its?= =?UTF-8?q?=20context?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit search --hits and trace hand out message ids, but nothing in the CLI could open one; drill-down meant the web page or raw grep (open since docs/devlog/2026-08-03-content-search-noise.org finding 4). render.WindowSession slices the wire-order message list around the target (exact id or unique prefix; ambiguity is an error, not a guess), detaches children so the window is exactly what it says, and never mutates the cached parse. `ccx view --at ID` renders that window, prints "message N of M" on stderr, and keeps the target even under --brief. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 2 + README.md | 1 + internal/cmd/view.go | 61 +++++++++++++++++++++++++++++- internal/render/window.go | 65 ++++++++++++++++++++++++++++++++ internal/render/window_test.go | 68 ++++++++++++++++++++++++++++++++++ skills/ccx/SKILL.md | 5 ++- 6 files changed, 199 insertions(+), 3 deletions(-) create mode 100644 internal/render/window.go create mode 100644 internal/render/window_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index e33d178..918d268 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,8 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ### Added - **Human interventions are first-class in `trace` and `log`.** "[Request interrupted by user]" (the human pressed stop) and permission-prompt rejections ("The user doesn't want to proceed with this tool use") are the human in the loop, but ccx read the first as a *prompt* — it opened a fake turn `u: [Request interrupted by user for tool use]` — and the second as an ordinary tool error. New parser kind `interrupt` (harness marker, never an exchange anchor) and `parser.IsToolDenial`; `trace` counts `interrupts` and `denials` on turn, step, and stats, marks the rejected call `denied` (not an error, it did not run), and badges them in the outline header and rows (`1 interrupt, 2 denied`; step `[4t 1! 1d]`); `log` reports kinds `interrupt` and `tool_denied` (`ccx log --scope today --all --kind interrupt,tool_denied`). 650 interruptions across 321 sessions and 145 rejections in one real store were invisible before. +- **`ccx view --at MESSAGE_ID [--context N]` walks from a citation to its context.** Search `--hits` and `trace` hand out message ids, but nothing in the CLI could open one; drill-down meant the web page or raw grep (open since docs/devlog/2026-08-03-content-search-noise.org finding 4). `--at` renders the cited message with N messages before and after it (wire order, flattened; the target survives `--brief`), and says where it sits: `message 1 of 763`. Prefixes resolve; ambiguous prefixes are an error, not a guess. + ### Changed - **`ccx trace` labels narration-less steps.** A step the agent never narrated (straight to tools) rendered as a bare badge row `1. [4t]`; the outline now says what it ran: `(no narration) Bash x3, Read` (docs/devlog/2026-08-17-codex-0147-rollout-drift.org finding 3). - **`ccx log --kind` and `--match PHRASE [-w]` turn the firehose into a timeline.** `log` emitted every record in scope (16k for one day) with no way to narrow; "what did the humans ask today, across every session" was not answerable. `--kind user_prompt,assistant_message` keeps only those kinds; `--match` keeps records whose raw transcript line contains the phrase (grep parity, `-w` for whole words) — the time-bounded complement of `search --hits`. Metrics stay honest: `records` is scope-wide, new `records_matched` is the narrowed count, `showing` is after `-n`. diff --git a/README.md b/README.md index 56d4cac..600b862 100644 --- a/README.md +++ b/README.md @@ -75,6 +75,7 @@ ccx sessions --provider=cx # Codex sessions in this workspace ccx sessions --after=2026-03-01 # Date filtered ccx sessions --scope yesterday --tz +8 --all --json # Session containers by end time ccx view [session] # View in terminal +ccx view [session] --at MSG_ID # Around one cited message (search --hits / trace message_id) ccx export --shape brief # Export conversation-only HTML ccx export --shape human # Only the human's turns, citable ccx trace [session] -o trace.json # Extract evidence for context folding diff --git a/internal/cmd/view.go b/internal/cmd/view.go index 5b44b35..793aeb1 100644 --- a/internal/cmd/view.go +++ b/internal/cmd/view.go @@ -27,7 +27,18 @@ SESSION can be: - Index: @1 (most recent), @2 (second most recent) - With project: myproject:e38536 -If SESSION is omitted, shows an interactive picker.`, +If SESSION is omitted, shows an interactive picker. + +--at MESSAGE_ID walks from a citation to its context: the message +with that id (a search --hits message_id, a trace step message_id; +prefixes work) plus --context N messages before and after it, +flattened. The target is always shown, even under --brief. + +Examples: + ccx view e38536 Whole session + ccx view e38536 --brief Conversation only + ccx view e38536 --at c8bd2144 Around one cited message + ccx view e38536 --at c8bd2144 --context 8`, Args: cobra.MaximumNArgs(1), RunE: runView, } @@ -40,6 +51,8 @@ var ( viewBrief bool viewAll bool viewColor string + viewAt string + viewContext int ) func init() { @@ -50,6 +63,8 @@ func init() { viewCmd.Flags().BoolVar(&viewFlat, "flat", false, "disable tree rendering") viewCmd.Flags().BoolVarP(&viewBrief, "brief", "b", false, "conversation only: human input, agent responses, compactions") viewCmd.Flags().StringVar(&viewColor, "color", "auto", "colorize output: auto, always, never") + viewCmd.Flags().StringVar(&viewAt, "at", "", "show the message with this id (or unique prefix) and its context") + viewCmd.Flags().IntVar(&viewContext, "context", 3, "with --at: messages of context before and after the target") } // resolveColorMode maps --color to a concrete decision; auto follows @@ -101,7 +116,17 @@ func runView(cmd *cobra.Command, args []string) error { return fmt.Errorf("failed to parse session: %w", err) } - if viewBrief { + if viewAt != "" { + window, index, total, err := render.WindowSession(fullSession, viewAt, viewContext, viewContext) + if err != nil { + return err + } + fmt.Fprintf(os.Stderr, "message %d of %d in %s · %d before / %d after\n", index, total, truncateID(fullSession.ID, 8), viewContext, viewContext) + if viewBrief { + window = briefKeeping(window, viewAt) + } + fullSession = window + } else if viewBrief { fullSession = render.BriefSession(fullSession) } @@ -121,6 +146,38 @@ func runView(cmd *cobra.Command, args []string) error { return render.Terminal(fullSession, opts) } +// briefKeeping applies the brief filter to a window but never drops +// the cited target itself: the point of --at is to see that message. +func briefKeeping(window *parser.Session, target string) *parser.Session { + brief := render.BriefSession(window) + for _, m := range brief.RootMessages { + if strings.HasPrefix(m.UUID, target) { + return brief + } + } + // Re-insert the target at its wire position among the kept ones. + var out []*parser.Message + inserted := false + for _, m := range window.RootMessages { + if strings.HasPrefix(m.UUID, target) { + out = append(out, m) + inserted = true + continue + } + for _, k := range brief.RootMessages { + if k.UUID == m.UUID { + out = append(out, k) + break + } + } + } + if !inserted { + return brief + } + brief.RootMessages = out + return brief +} + func sessionLookupQuery(projectName string, all bool) (catalog.SessionQuery, error) { if projectName != "" { return catalog.SessionQuery{ diff --git a/internal/render/window.go b/internal/render/window.go new file mode 100644 index 0000000..0e8d344 --- /dev/null +++ b/internal/render/window.go @@ -0,0 +1,65 @@ +package render + +import ( + "fmt" + "strings" + + "github.com/thevibeworks/ccx/internal/parser" +) + +// WindowSession returns a copy of session holding only the message +// whose UUID starts with target plus `before` messages before it and +// `after` after it, in wire order, flattened (children detached so +// the window is exactly what it says). This is the walk from a +// citation — a search hit's message_id, a trace step's message_id — +// back to its surrounding context without leaving ccx. index/total +// locate the target in the full session for the caller's header. +func WindowSession(session *parser.Session, target string, before, after int) (*parser.Session, int, int, error) { + if session == nil { + return nil, 0, 0, fmt.Errorf("no session") + } + target = strings.TrimSpace(target) + if target == "" { + return nil, 0, 0, fmt.Errorf("--at needs a message id (or unique prefix)") + } + all := parser.FlattenSessionMessages(session) + idx := -1 + for i, m := range all { + if m == nil || !strings.HasPrefix(m.UUID, target) { + continue + } + if m.UUID == target { + idx = i + break + } + if idx >= 0 && all[idx].UUID != m.UUID { + return nil, 0, 0, fmt.Errorf("message id prefix %q is ambiguous (%s, %s, ...); use more characters", target, all[idx].UUID, m.UUID) + } + idx = i + } + if idx < 0 { + return nil, 0, 0, fmt.Errorf("no message with id prefix %q in session %s", target, session.ID) + } + if before < 0 { + before = 0 + } + if after < 0 { + after = 0 + } + start := idx - before + if start < 0 { + start = 0 + } + end := idx + after + 1 + if end > len(all) { + end = len(all) + } + out := *session + out.RootMessages = make([]*parser.Message, 0, end-start) + for _, m := range all[start:end] { + flat := *m + flat.Children = nil + out.RootMessages = append(out.RootMessages, &flat) + } + return &out, idx + 1, len(all), nil +} diff --git a/internal/render/window_test.go b/internal/render/window_test.go new file mode 100644 index 0000000..0ad8337 --- /dev/null +++ b/internal/render/window_test.go @@ -0,0 +1,68 @@ +package render + +import ( + "strings" + "testing" + + "github.com/thevibeworks/ccx/internal/parser" +) + +func winMsg(uuid string, children ...*parser.Message) *parser.Message { + return &parser.Message{UUID: uuid, Kind: parser.KindAssistant, Type: "assistant", Children: children} +} + +// The window is a wire-order slice around the target, flattened +// (children detached), clamped at both ends; the target is found by +// exact id or unique prefix; ambiguity and misses are errors. +func TestWindowSession(t *testing.T) { + // Tree: a -> b -> c -> d -> e (linear chain), plus f as a second root. + e := winMsg("eeee-5") + d := winMsg("dddd-4", e) + c := winMsg("cccc-3", d) + b := winMsg("bbbb-2", c) + a := winMsg("aaaa-1", b) + f := winMsg("aaab-6") + session := &parser.Session{ID: "s", RootMessages: []*parser.Message{a, f}} + + win, idx, total, err := WindowSession(session, "cccc", 1, 1) + if err != nil { + t.Fatal(err) + } + if idx != 3 || total != 6 { + t.Fatalf("index/total: %d/%d", idx, total) + } + got := []string{} + for _, m := range win.RootMessages { + got = append(got, m.UUID) + if len(m.Children) != 0 { + t.Fatalf("children must be detached in the window: %s", m.UUID) + } + } + if strings.Join(got, ",") != "bbbb-2,cccc-3,dddd-4" { + t.Fatalf("window: %v", got) + } + // Original tree untouched. + if len(session.RootMessages[0].Children) != 1 { + t.Fatal("window must not mutate the parsed session") + } + + // Clamp at the start; big context. + win, _, _, err = WindowSession(session, "aaaa-1", 5, 1) + if err != nil || len(win.RootMessages) != 2 || win.RootMessages[0].UUID != "aaaa-1" { + t.Fatalf("clamped window: %+v %v", win.RootMessages, err) + } + // Ambiguous prefix "aaa" matches aaaa-1 and aaab-6. + if _, _, _, err := WindowSession(session, "aaa", 1, 1); err == nil || !strings.Contains(err.Error(), "ambiguous") { + t.Fatalf("ambiguous prefix must error, got %v", err) + } + // Exact id wins even when it is also a prefix of another id. + if _, idx, _, err := WindowSession(session, "aaaa-1", 0, 0); err != nil || idx != 1 { + t.Fatalf("exact id: %d %v", idx, err) + } + if _, _, _, err := WindowSession(session, "zzzz", 1, 1); err == nil { + t.Fatal("missing id must error") + } + if _, _, _, err := WindowSession(session, "", 1, 1); err == nil { + t.Fatal("empty target must error") + } +} diff --git a/skills/ccx/SKILL.md b/skills/ccx/SKILL.md index afdec8d..8b19422 100644 --- a/skills/ccx/SKILL.md +++ b/skills/ccx/SKILL.md @@ -25,6 +25,7 @@ ccx │ └── --goal SLUG # Filter by launch-receipt goal (deva --goal) ├── view [session] # View session in terminal │ └── --brief # Conversation only +│ └── --at MSG_ID [--context N] # Walk from a citation (search --hits / trace message_id) to its context ├── export [session] # Export session │ └── --format html|md|org|exec │ └── --shape full|brief|trace|exchange|human @@ -183,7 +184,9 @@ ccx search --raw "deploy" # Grep parity over raw transcript lines `-w` matters when the term prefixes a common word ("semantica" vs "semantically"); `--json` carries `matches`, `previews`, `first_hit`. Cite from `--hits --json` (`message_id`, `time`, `quote`) rather than -from a session-level count when a claim needs evidence. +from a session-level count when a claim needs evidence; walk to the +context with `ccx view --at ` (web: +`/session//#msg-`). Web search supports provider prefixes: `cc: auth bug`, `cx: codex query`, `gx: grok query` From bb9f07f826196e7adbddfc81042936791d0f83f0 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 06:04:07 -0700 Subject: [PATCH 07/13] feat(search): prompt history as a --content source (type prompt) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude Code and Codex append every human prompt to history.jsonl, and those files outlive session cleanup — in one real store prompts reach back to 2025-09-28 while the oldest session file is 2025-12-07. For "when did we first say X" that is the longest-lived evidence there is. --content now scans both history files (formats differ: claude display/project/sessionId/timestamp-ms vs codex session_id/ts/text). Only prompts whose session id is not in the store surface, so a prompt is cited once — from the session while it exists, from history after cleanup. Rows are type `prompt` with FIRST, a [user] quote, and under --hits a history:line anchor; --sort first interleaves them into the timeline. Live: `search --content -w --sort first temporal_chaggr` -> first mention 2025-09-28 00:54 from history (sessions long gone). Tests: TestScanPromptHistory (both formats, known-session skip, anonymous prompts, missing file, result shape). Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 1 + internal/cmd/search.go | 22 +++- internal/cmd/search_history.go | 198 +++++++++++++++++++++++++++++++++ internal/cmd/search_test.go | 60 ++++++++++ skills/ccx/SKILL.md | 4 + 5 files changed, 283 insertions(+), 2 deletions(-) create mode 100644 internal/cmd/search_history.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 918d268..12dac50 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ### Added - **Human interventions are first-class in `trace` and `log`.** "[Request interrupted by user]" (the human pressed stop) and permission-prompt rejections ("The user doesn't want to proceed with this tool use") are the human in the loop, but ccx read the first as a *prompt* — it opened a fake turn `u: [Request interrupted by user for tool use]` — and the second as an ordinary tool error. New parser kind `interrupt` (harness marker, never an exchange anchor) and `parser.IsToolDenial`; `trace` counts `interrupts` and `denials` on turn, step, and stats, marks the rejected call `denied` (not an error, it did not run), and badges them in the outline header and rows (`1 interrupt, 2 denied`; step `[4t 1! 1d]`); `log` reports kinds `interrupt` and `tool_denied` (`ccx log --scope today --all --kind interrupt,tool_denied`). 650 interruptions across 321 sessions and 145 rejections in one real store were invisible before. +- **`ccx search --content` scans prompt history.** Claude Code and Codex append every human prompt to `history.jsonl`, and those files outlive session cleanup (one real store: prompts back to 2025-09-28, sessions back to 2025-12-07). Matches surface as type `prompt` — only for prompts whose session file is no longer in the store, so a prompt never appears twice — with `FIRST`, a `[user]` quote, and under `--hits` a `history:line` anchor. "When did we first say X" now reaches past the session horizon. - **`ccx view --at MESSAGE_ID [--context N]` walks from a citation to its context.** Search `--hits` and `trace` hand out message ids, but nothing in the CLI could open one; drill-down meant the web page or raw grep (open since docs/devlog/2026-08-03-content-search-noise.org finding 4). `--at` renders the cited message with N messages before and after it (wire order, flattened; the target survives `--brief`), and says where it sits: `message 1 of 763`. Prefixes resolve; ambiguous prefixes are an error, not a guess. ### Changed diff --git a/internal/cmd/search.go b/internal/cmd/search.go index 90a4a97..16a9409 100644 --- a/internal/cmd/search.go +++ b/internal/cmd/search.go @@ -39,7 +39,10 @@ With --content, also scan conversation text inside session files (including subagent files): user prompts and assistant replies, ranked by hit count with a matched-text preview and the time of the earliest match (FIRST). Injected noise — tool results, hook -attachments, command echoes — doesn't count. +attachments, command echoes — doesn't count. Prompt history +(~/.claude/history.jsonl, ~/.codex/history.jsonl) is scanned too and +surfaces as type "prompt" for prompts whose session file is gone — +the longest-lived evidence, past session cleanup. Add --raw to match every raw transcript line instead: grep parity, no parse, misses nothing grep would find. @@ -349,6 +352,17 @@ func runSearch(cmd *cobra.Command, args []string) error { results = append(results, *r) } + // Prompt history: prompts whose session file is gone (cleanup) + // still exist here; the only place "when did we first say X" can + // be answered past the session horizon. + if searchContent && searchType != "project" { + settings := config.Load() + known := knownSessionIDs(candidates) + for _, h := range scanPromptHistory(promptHistoryFiles(settings.ClaudeHome, settings.CodexHome), m, known) { + results = append(results, promptResult(h)) + } + } + if searchHits { return printSearchHits(collectHits(results)) } @@ -744,7 +758,11 @@ func printSearchHits(hits []searchHit) error { if id == "" { id = "-" } - fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", t, h.Session, h.Role, truncateID(id, 8), cleanDisplayText(h.Quote)) + session := h.Session + if session == "" { + session = "-" + } + fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", t, session, h.Role, truncateID(id, 8), cleanDisplayText(h.Quote)) } return w.Flush() } diff --git a/internal/cmd/search_history.go b/internal/cmd/search_history.go new file mode 100644 index 0000000..ea4842b --- /dev/null +++ b/internal/cmd/search_history.go @@ -0,0 +1,198 @@ +package cmd + +import ( + "bufio" + "encoding/json" + "fmt" + "io" + "os" + "path/filepath" + "strings" + "time" +) + +// Prompt history as a search source. Claude Code and Codex append every +// human prompt to a history file that outlives session cleanup, so it +// is the longest-lived evidence for "when did we first say X". Only +// prompts whose session is no longer in the store surface here — when +// the session file exists its own content hit is the better citation, +// and one prompt must not appear twice in a timeline. + +// promptHit is one matching prompt from a history file. +type promptHit struct { + Provider string + Project string // workspace basename when the file records one + SessionID string + Path string + Line int + Time time.Time + Matches int + Quote string +} + +// promptHistoryFiles lists the history files to scan for the given +// provider homes (missing files are simply absent). +func promptHistoryFiles(claudeHome, codexHome string) map[string]string { + files := map[string]string{} + if claudeHome != "" { + files["claude-code"] = filepath.Join(claudeHome, "history.jsonl") + } + if codexHome != "" { + files["codex"] = filepath.Join(codexHome, "history.jsonl") + } + return files +} + +// scanPromptHistory returns matching prompts across the history files, +// skipping entries whose session id is in knownSessions. +func scanPromptHistory(files map[string]string, m textMatcher, knownSessions map[string]bool) []promptHit { + var hits []promptHit + for _, provider := range []string{"claude-code", "codex"} { + path, ok := files[provider] + if !ok { + continue + } + hits = append(hits, scanPromptHistoryFile(provider, path, m, knownSessions)...) + } + return hits +} + +func scanPromptHistoryFile(provider, path string, m textMatcher, knownSessions map[string]bool) []promptHit { + file, err := os.Open(path) + if err != nil { + return nil // no history is normal + } + defer file.Close() + + var hits []promptHit + reader := bufio.NewReaderSize(file, 64*1024) + lineNo := 0 + for { + line, err := reader.ReadString('\n') + if line != "" { + lineNo++ + // Cheap literal prefilter on the raw line; the decoded + // text decides. + if rawPrefilterSafe(m.query) && !m.literal().matches(line) { + goto next + } + if hit, ok := decodePromptLine(provider, line); ok { + if hit.SessionID != "" && knownSessions[hit.SessionID] { + goto next + } + if n := m.count(hit.text); n > 0 { + idx, qlen := m.index(hit.text) + hits = append(hits, promptHit{ + Provider: provider, + Project: hit.project, + SessionID: hit.SessionID, + Path: path, + Line: lineNo, + Time: hit.time, + Matches: n, + Quote: matchSnippet(hit.text, idx, qlen), + }) + } + } + } + next: + if err != nil { + if err != io.EOF { + fmt.Fprintf(os.Stderr, "warning: read error in %s: %v\n", filepath.Base(path), err) + } + return hits + } + } +} + +type promptLine struct { + SessionID string + project string + time time.Time + text string +} + +// decodePromptLine understands both history formats: +// +// claude-code: {"display":"...","project":"/path","sessionId":"...","timestamp":} +// codex: {"session_id":"...","ts":,"text":"..."} +func decodePromptLine(provider, line string) (promptLine, bool) { + switch provider { + case "claude-code": + var rec struct { + Display string `json:"display"` + Project string `json:"project"` + SessionID string `json:"sessionId"` + Timestamp int64 `json:"timestamp"` + } + if err := json.Unmarshal([]byte(line), &rec); err != nil || rec.Display == "" { + return promptLine{}, false + } + out := promptLine{SessionID: rec.SessionID, text: rec.Display} + if rec.Project != "" { + out.project = filepath.Base(rec.Project) + } + if rec.Timestamp > 0 { + out.time = time.UnixMilli(rec.Timestamp).UTC() + } + return out, true + case "codex": + var rec struct { + SessionID string `json:"session_id"` + TS int64 `json:"ts"` + Text string `json:"text"` + } + if err := json.Unmarshal([]byte(line), &rec); err != nil || rec.Text == "" { + return promptLine{}, false + } + out := promptLine{SessionID: rec.SessionID, text: rec.Text} + if rec.TS > 0 { + out.time = time.Unix(rec.TS, 0).UTC() + } + return out, true + } + return promptLine{}, false +} + +// promptResult renders a prompt hit as a search result row. +func promptResult(h promptHit) searchResult { + project := h.Project + if project == "" { + project = "(" + h.Provider + " history)" + } + session := "" + if h.SessionID != "" { + session = truncateID(h.SessionID, 8) + } + res := searchResult{ + Type: "prompt", + Project: project, + Session: session, + Path: h.Path, + Summary: fmt.Sprintf("%d hits · [user] %s", h.Matches, truncateDisplay(h.Quote, 56)), + Time: "-", + Matches: h.Matches, + Previews: []contentPreview{{Role: "user", Text: h.Quote}}, + Priority: 4, + firstHit: h.Time, + hits: []searchHit{{ + Project: project, Session: session, Path: h.Path, MessageID: fmt.Sprintf("line:%d", h.Line), + Time: h.Time, Role: "user", Matches: h.Matches, Quote: h.Quote, + }}, + } + if !h.Time.IsZero() { + res.FirstHit = h.Time.UTC().Format(time.RFC3339) + } + return res +} + +// knownSessionIDs collects every session id the store currently holds. +func knownSessionIDs(sessions []sessionCandidate) map[string]bool { + known := make(map[string]bool, len(sessions)) + for _, c := range sessions { + if c.session != nil && c.session.ID != "" { + known[strings.ToLower(c.session.ID)] = true + } + } + return known +} diff --git a/internal/cmd/search_test.go b/internal/cmd/search_test.go index 54a5870..04167e2 100644 --- a/internal/cmd/search_test.go +++ b/internal/cmd/search_test.go @@ -484,3 +484,63 @@ func TestSessionSearcherHits(t *testing.T) { t.Fatalf("collectHits order: %v", ids) } } + +// Prompt history is the search source that outlives session cleanup: +// both providers' formats decode, only prompts whose session is not in +// the store surface (no double citations), and each hit carries a +// file:line anchor and time. +func TestScanPromptHistory(t *testing.T) { + dir := t.TempDir() + claude := filepath.Join(dir, "claude-history.jsonl") + codex := filepath.Join(dir, "codex-history.jsonl") + if err := os.WriteFile(claude, []byte(strings.Join([]string{ + `{"display":"tell me about semantica","pastedContents":{},"timestamp":1759046069307,"project":"/Users/x/wrk/old-proj","sessionId":"gone-1111"}`, + `{"display":"semantically fine, ignore","timestamp":1759046070000,"project":"/Users/x/wrk/old-proj","sessionId":"gone-2222"}`, + `{"display":"semantica again, still in store","timestamp":1759046071000,"project":"/Users/x/wrk/live","sessionId":"live-3333"}`, + `{"display":"no session id but semantica","timestamp":1759046072000,"project":"/Users/x/wrk/anon"}`, + `not json`, + }, "\n")+"\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(codex, []byte(strings.Join([]string{ + `{"session_id":"cx-gone","ts":1754596435,"text":"Semantica in codex"}`, + `{"session_id":"cx-live","ts":1754596436,"text":"semantica in a live codex session"}`, + }, "\n")+"\n"), 0o644); err != nil { + t.Fatal(err) + } + files := map[string]string{"claude-code": claude, "codex": codex} + known := map[string]bool{"live-3333": true, "cx-live": true} + + hits := scanPromptHistory(files, word("semantica"), known) + if len(hits) != 3 { + t.Fatalf("hits: got %d, want 3 (gone-1111, anon, cx-gone): %+v", len(hits), hits) + } + h := hits[0] + if h.Provider != "claude-code" || h.Project != "old-proj" || h.SessionID != "gone-1111" || h.Line != 1 || h.Matches != 1 { + t.Fatalf("first hit: %+v", h) + } + if !h.Time.Equal(time.UnixMilli(1759046069307).UTC()) || !strings.Contains(h.Quote, "semantica") { + t.Fatalf("first hit time/quote: %+v", h) + } + if hits[1].SessionID != "" || hits[1].Project != "anon" { + t.Fatalf("anonymous prompt: %+v", hits[1]) + } + if hits[2].Provider != "codex" || hits[2].SessionID != "cx-gone" || !hits[2].Time.Equal(time.Unix(1754596435, 0).UTC()) { + t.Fatalf("codex hit: %+v", hits[2]) + } + // Substring mode also takes "semantically". + if got := len(scanPromptHistory(files, sub("semantica"), known)); got != 4 { + t.Fatalf("substring hits: got %d, want 4", got) + } + // Missing files are not an error. + if got := scanPromptHistory(map[string]string{"claude-code": filepath.Join(dir, "nope.jsonl")}, sub("x"), nil); len(got) != 0 { + t.Fatalf("missing history: %+v", got) + } + + // As a result row: type prompt, priority after content, first-hit + // set, one citation hit with a line anchor. + r := promptResult(hits[0]) + if r.Type != "prompt" || r.Priority != 4 || r.FirstHit == "" || len(r.hits) != 1 || r.hits[0].MessageID != "line:1" || r.hits[0].Role != "user" { + t.Fatalf("prompt result: %+v", r) + } +} diff --git a/skills/ccx/SKILL.md b/skills/ccx/SKILL.md index 8b19422..ebcfbdd 100644 --- a/skills/ccx/SKILL.md +++ b/skills/ccx/SKILL.md @@ -183,6 +183,10 @@ ccx search --raw "deploy" # Grep parity over raw transcript lines `-w` matters when the term prefixes a common word ("semantica" vs "semantically"); `--json` carries `matches`, `previews`, `first_hit`. +`--content` also scans prompt history (`~/.claude/history.jsonl`, +`~/.codex/history.jsonl`): type `prompt` rows are prompts whose +session file is gone — the only evidence past session cleanup, so +"when did we first say X" reaches back further than the sessions do. Cite from `--hits --json` (`message_id`, `time`, `quote`) rather than from a session-level count when a claim needs evidence; walk to the context with `ccx view --at ` (web: From c0d07a57bcd0be11e3b09f726f32cc7d13346512 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 06:04:40 -0700 Subject: [PATCH 08/13] docs(skills): recap/retro use interventions, related, view --at; 0005 roadmap status Co-Authored-By: Claude Fable 5 --- ...idence-citations-lessons-from-semantica.md | 35 ++++++++++--------- skills/ccx-recap/SKILL.md | 12 +++++-- skills/ccx-retro/SKILL.md | 9 +++-- 3 files changed, 35 insertions(+), 21 deletions(-) diff --git a/docs/design/0005-evidence-citations-lessons-from-semantica.md b/docs/design/0005-evidence-citations-lessons-from-semantica.md index d077592..8eef044 100644 --- a/docs/design/0005-evidence-citations-lessons-from-semantica.md +++ b/docs/design/0005-evidence-citations-lessons-from-semantica.md @@ -54,20 +54,23 @@ principles we adopt, how each maps onto ccx, and what we reject. (~10x) with progress; `-n` shorthand on `sessions`/`projects`/`log`. Devlog: `docs/devlog/2026-08-18-search-word-boundary-dogfood.org`. -## Roadmap (small, ordered) +## Roadmap (small, ordered) — status 2026-08-18 -1. `ccx log --match QUERY [-w]`: records containing a term inside a time - window — the bi-temporal slice (`search --hits` is unbounded in - time; `log` is bounded but cannot filter by term). Reuse - `textMatcher`. -2. `ccx view --at ` (or `--grep`): walk from a - citation to its surrounding context without leaving ccx (open since - `docs/devlog/2026-08-03-content-search-noise.org` finding 4). -3. `~/.claude/history.jsonl` as a `type: prompt` search source: the - longest-lived evidence (prompts back to 2025-09) for "when did we - first say X" once session files have been cleaned up. -4. Session lineage: `fork`/`resume` parents as derives-from edges in - `sessions --json` and `trace`, so a decision chain can cross - session boundaries. -5. Readability contract test for `trace`: every step carries - human-readable narration or a mutation summary, never only ids. +1. DONE `ccx log --match QUERY [-w]` + `--kind`: records containing a + term inside a time window — the bi-temporal slice (`search --hits` + is unbounded in time). Commit afa3785. +2. DONE `ccx view --at [--context N]`: walk from + a citation to its context. Commit 8d8f77a. +3. DONE prompt history (`~/.claude/history.jsonl`, `~/.codex/history.jsonl`) + as a `type: prompt` search source, deduplicated against live + sessions. Commit bb9f07f. +4. DONE session connections: `ccx related` and `related` in + `trace --full` (docs/design/0006-session-connections.md). Commit + 8819774. Human interventions (interrupts/denials) as first-class + trace/log facts: 3a8644f. +5. DONE `trace` labels narration-less steps ("(no narration) Bash x3, + Read"); a step never renders as a bare badge row. 3a8644f. + +Next candidates: web session page shows `related` and intervention +badges (needs a visual pass); Codex approval rejections as denials; +`sessions --json` carrying interrupts/denials from quick parse. diff --git a/skills/ccx-recap/SKILL.md b/skills/ccx-recap/SKILL.md index 181ae03..ba72262 100644 --- a/skills/ccx-recap/SKILL.md +++ b/skills/ccx-recap/SKILL.md @@ -40,10 +40,16 @@ its own CLI surface. fits; read it whole. Steps are the agent's own narration at the moment it acted — that sequence IS the story skeleton. 2. **Pick what matters.** Turns with edits, errors, linked commits, - high cost, or user pushback. Ignore command noise (`is_command`). + high cost, user pushback, or human interventions (`interrupts` — + the human pressed stop; `denials` — a tool call refused at the + permission prompt; both are on turns, steps, and stats). Ignore + command noise (`is_command`). 3. **Drill only there.** `ccx trace --turn N` for full narration, - mutations, and error attribution. Raw text when needed: the - `anchor_id` / `message_id` point into the session file. + mutations, and error attribution. Raw text when needed: `ccx view + --at ` opens the cited message in context. + When the story crosses sessions (a handoff picked up, a fork, a + parallel agent), `ccx related ` says which sessions and + how, with evidence — cite those anchors, do not guess the link. 4. **Verify before claiming.** "Done" requires evidence: a passing test step, a linked commit, a verification narration. Tag every important claim: diff --git a/skills/ccx-retro/SKILL.md b/skills/ccx-retro/SKILL.md index 55bd6cb..a8dcf18 100644 --- a/skills/ccx-retro/SKILL.md +++ b/skills/ccx-retro/SKILL.md @@ -32,12 +32,17 @@ matching its own CLI surface. - turns where the user pushed back — read the actual `user_text` and judge with your own comprehension; there is no correction flag, because keyword matching lies + - `interrupts` and `denials` (turn/step/stats): the human pressed + stop or refused a tool call — the sharpest correction signal in + the log; the denied call's evidence is marked `denied` - cost spikes without matching edits (spinning) - `warnings` (evidence gaps are findings too) -2. **Drill.** `ccx trace --turn N` on each suspect. Reconstruct: +2. **Drill.** `ccx trace --turn N` on each suspect; `ccx view + --at ` for the exact exchange. Reconstruct: what did the agent believe, what was actually true, what evidence was available at the time, what finally corrected it (user, test, - self-check)? + self-check)? If the mistake was inherited from an earlier session + (a stale handoff, a fork), `ccx related ` finds it. 3. **Classify each finding.** - `mistake` — agent had the evidence and got it wrong - `friction` — environment/tooling failed the agent From 0f79cbaad5e85d0aa85b90056bc8b1ed71f57822 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 06:06:05 -0700 Subject: [PATCH 09/13] feat(web): GET /api/related//; RelateWorkspace in trace MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Move the workspace relation (list every session of the anchor's workspace, profile on a worker pool, relate) from the CLI into trace.RelateWorkspace over a narrow SessionSource interface, so the CLI, the web API, and tests share one implementation. The CLI wrapper keeps its stderr progress. New endpoint returns the ccx.related.v1 envelope (related, total, shown, warnings; ?limit=N) — the same shape as `ccx related --json` — so a session-page panel or an agent reading the API sees exactly what the CLI prints. On demand only: it costs a parse of every workspace session (cached after the first call). Test: TestHandleAPIRelated (builds_on + previous between two workspace sessions, 404s for unknown session and malformed path). Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 1 + docs/schema.md | 2 +- internal/cmd/related.go | 75 +++----------------- internal/trace/related_workspace.go | 104 ++++++++++++++++++++++++++++ internal/web/server.go | 49 +++++++++++++ internal/web/server_test.go | 69 ++++++++++++++++++ 6 files changed, 233 insertions(+), 67 deletions(-) create mode 100644 internal/trace/related_workspace.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 12dac50..50e819c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ### Added - **Human interventions are first-class in `trace` and `log`.** "[Request interrupted by user]" (the human pressed stop) and permission-prompt rejections ("The user doesn't want to proceed with this tool use") are the human in the loop, but ccx read the first as a *prompt* — it opened a fake turn `u: [Request interrupted by user for tool use]` — and the second as an ordinary tool error. New parser kind `interrupt` (harness marker, never an exchange anchor) and `parser.IsToolDenial`; `trace` counts `interrupts` and `denials` on turn, step, and stats, marks the rejected call `denied` (not an error, it did not run), and badges them in the outline header and rows (`1 interrupt, 2 denied`; step `[4t 1! 1d]`); `log` reports kinds `interrupt` and `tool_denied` (`ccx log --scope today --all --kind interrupt,tool_denied`). 650 interruptions across 321 sessions and 145 rejections in one real store were invisible before. +- **`GET /api/related//` serves session connections as JSON** — the `ccx.related.v1` envelope (`related`, `total`, `shown`, `warnings`; `?limit=N`), computed by the same `trace.RelateWorkspace` the CLI uses, so a web panel or an agent reading the API sees exactly what `ccx related --json` prints. Fetched on demand: it costs a parse of every workspace session (cached after the first call). - **`ccx search --content` scans prompt history.** Claude Code and Codex append every human prompt to `history.jsonl`, and those files outlive session cleanup (one real store: prompts back to 2025-09-28, sessions back to 2025-12-07). Matches surface as type `prompt` — only for prompts whose session file is no longer in the store, so a prompt never appears twice — with `FIRST`, a `[user]` quote, and under `--hits` a `history:line` anchor. "When did we first say X" now reaches past the session horizon. - **`ccx view --at MESSAGE_ID [--context N]` walks from a citation to its context.** Search `--hits` and `trace` hand out message ids, but nothing in the CLI could open one; drill-down meant the web page or raw grep (open since docs/devlog/2026-08-03-content-search-noise.org finding 4). `--at` renders the cited message with N messages before and after it (wire order, flattened; the target survives `--brief`), and says where it sits: `message 1 of 763`. Prefixes resolve; ambiguous prefixes are an error, not a guess. diff --git a/docs/schema.md b/docs/schema.md index 41776e1..7dcfd93 100644 --- a/docs/schema.md +++ b/docs/schema.md @@ -15,7 +15,7 @@ they do not know. | `ccx.outline.v1` | `ccx trace --json` | Session skeleton: every turn and step headline with rollups. Read this first; it always fits. | | `ccx.turn.v1` | `ccx trace --turn N` | One turn with full step evidence, plus the sidechain entries that turn references, plus warnings. | | `ccx.trace.v2` | `ccx trace --full` | Complete evidence bundle: all turns/steps, sidechains, git correlation, workspace context, `related` sessions, stats, warnings. Large. | -| `ccx.related.v1` | `ccx related --json` | The anchor session's connections to the other sessions of its workspace: `related[]` of `{session_id, provider, summary, start, end, strength, relations[]}`, plus `total`/`shown`. Each relation is `{kind, count?, paths?, evidence[], truncated?}`; evidence items are `{session_id, message_id, time, path?, quote?}`. Kinds: `forked_from`/`fork_of`, `mentions`/`mentioned_by`, `handoff_from`/`handoff_to`, `builds_on`/`built_on_by`, `overlaps`, `previous`/`next`. Strength is `strong`/`medium`/`weak`. | +| `ccx.related.v1` | `ccx related --json`, `GET /api/related//` | The anchor session's connections to the other sessions of its workspace: `related[]` of `{session_id, provider, summary, start, end, strength, relations[]}`, plus `total`/`shown`. Each relation is `{kind, count?, paths?, evidence[], truncated?}`; evidence items are `{session_id, message_id, time, path?, quote?}`. Kinds: `forked_from`/`fork_of`, `mentions`/`mentioned_by`, `handoff_from`/`handoff_to`, `builds_on`/`built_on_by`, `overlaps`, `previous`/`next`. Strength is `strong`/`medium`/`weak`. | | `ccx.log.v1` | `ccx log --json`, `ccx insight --json` | Time-scoped records across sessions with pre-computed `days[]` / `providers[]` / `workspaces[]` aggregates. Record `kind` is provider-normalized: `user_prompt` and `assistant_message` are the visible conversation only (Claude command markers/echoes/notifications are `command`/`command_output`/`notification`, injected meta is `meta`, compaction carriers `compact_summary`; Codex 0.147 raw `response_item` messages are `model_input`/`model_output` and duplicated legacy events `legacy_message`). With `--kind`/`--match`, `metrics.records` stays scope-wide and `metrics.records_matched` reports the narrowed count before any limit. | ## Versioning policy diff --git a/internal/cmd/related.go b/internal/cmd/related.go index 6125591..8a8875a 100644 --- a/internal/cmd/related.go +++ b/internal/cmd/related.go @@ -6,7 +6,6 @@ import ( "os" "path/filepath" "strings" - "sync" "text/tabwriter" "time" @@ -120,78 +119,22 @@ func runRelated(cmd *cobra.Command, args []string) error { return printRelated(session, related, total) } -// relateSession profiles every session of the anchor's workspace and -// returns the anchor's connections. Shared by `ccx related` and the -// trace --full bundle. +// relateSession is the CLI wrapper over trace.RelateWorkspace: same +// worker pool as search, progress on stderr when it is a terminal. func relateSession(backend provider.Backend, anchor *parser.Session) ([]trace.RelatedSession, []trace.TraceWarning, error) { + // Progress needs the total; list once, cheaply, for the count. query := catalog.SessionQuery{Scope: catalog.ScopeProject, ProjectName: anchor.ProjectName} if strings.TrimSpace(anchor.CWD) != "" { query = catalog.SessionQuery{Scope: catalog.ScopeWorkspace, WorkspacePath: anchor.CWD} } - sessions, err := backend.ListSessions(query.WithoutLimit().WithoutProviderFilter()) - if err != nil { - return nil, nil, fmt.Errorf("list workspace sessions: %w", err) - } - // The anchor may be missing from the workspace listing when its - // cwd differs from the project path (a fork into another dir); - // it always takes part. - found := false - for _, s := range sessions { - if s.FilePath == anchor.FilePath { - found = true - break - } - } - if !found { - sessions = append(sessions, anchor) - } - - profiles := make([]*trace.SessionProfile, len(sessions)) - var warnings []trace.TraceWarning - var warnMu sync.Mutex - progress := newScanProgress(len(sessions), true) - var wg sync.WaitGroup - next := make(chan int) - for w := 0; w < searchWorkers(true); w++ { - wg.Add(1) - go func() { - defer wg.Done() - for i := range next { - full, err := backend.ParseSession(sessions[i].FilePath) - if err != nil { - warnMu.Lock() - warnings = append(warnings, trace.TraceWarning{Kind: "related_parse_failed", Message: fmt.Sprintf("skipping %s: %v", filepath.Base(sessions[i].FilePath), err)}) - warnMu.Unlock() - } else { - profiles[i] = trace.ProfileSession(full) - } - progress.tick() - } - }() + total := 1 + if sessions, err := backend.ListSessions(query.WithoutLimit().WithoutProviderFilter()); err == nil { + total = len(sessions) + 1 } - for i := range sessions { - next <- i - } - close(next) - wg.Wait() + progress := newScanProgress(total, true) + related, warnings, err := trace.RelateWorkspace(backend, anchor, searchWorkers(true), progress.tick) progress.done() - - var anchorProfile *trace.SessionProfile - others := make([]*trace.SessionProfile, 0, len(profiles)) - for i, p := range profiles { - if p == nil { - continue - } - if sessions[i].FilePath == anchor.FilePath { - anchorProfile = p - continue - } - others = append(others, p) - } - if anchorProfile == nil { - return nil, warnings, fmt.Errorf("could not parse anchor session %s", anchor.ID) - } - return trace.RelateSessions(anchorProfile, others), warnings, nil + return related, warnings, err } func printRelated(anchor *parser.Session, related []trace.RelatedSession, total int) error { diff --git a/internal/trace/related_workspace.go b/internal/trace/related_workspace.go new file mode 100644 index 0000000..7ddd0f6 --- /dev/null +++ b/internal/trace/related_workspace.go @@ -0,0 +1,104 @@ +package trace + +import ( + "fmt" + "path/filepath" + "strings" + "sync" + + "github.com/thevibeworks/ccx/internal/catalog" + "github.com/thevibeworks/ccx/internal/parser" +) + +// SessionSource is the slice of provider.Backend that workspace +// relation needs; narrowed so the CLI, the web API, and tests share +// one implementation. +type SessionSource interface { + ListSessions(query catalog.SessionQuery) ([]*parser.Session, error) + ParseSession(filePath string) (*parser.Session, error) +} + +// RelateWorkspace profiles every session of the anchor's workspace +// (all providers) on `workers` goroutines and returns the anchor's +// connections. tick, when non-nil, is called once per session +// profiled (progress). Sessions that fail to parse become warnings, +// never silent gaps. +func RelateWorkspace(src SessionSource, anchor *parser.Session, workers int, tick func()) ([]RelatedSession, []TraceWarning, error) { + if src == nil || anchor == nil { + return nil, nil, fmt.Errorf("no session") + } + query := catalog.SessionQuery{Scope: catalog.ScopeProject, ProjectName: anchor.ProjectName} + if strings.TrimSpace(anchor.CWD) != "" { + query = catalog.SessionQuery{Scope: catalog.ScopeWorkspace, WorkspacePath: anchor.CWD} + } + sessions, err := src.ListSessions(query.WithoutLimit().WithoutProviderFilter()) + if err != nil { + return nil, nil, fmt.Errorf("list workspace sessions: %w", err) + } + // The anchor may be missing from the workspace listing when its + // cwd differs from the project path (a fork into another dir); + // it always takes part. + found := false + for _, s := range sessions { + if s.FilePath == anchor.FilePath { + found = true + break + } + } + if !found { + sessions = append(sessions, anchor) + } + if workers < 1 { + workers = 1 + } + if workers > len(sessions) { + workers = len(sessions) + } + + profiles := make([]*SessionProfile, len(sessions)) + var warnings []TraceWarning + var warnMu sync.Mutex + var wg sync.WaitGroup + next := make(chan int) + for w := 0; w < workers; w++ { + wg.Add(1) + go func() { + defer wg.Done() + for i := range next { + full, err := src.ParseSession(sessions[i].FilePath) + if err != nil { + warnMu.Lock() + warnings = append(warnings, TraceWarning{Kind: "related_parse_failed", Message: fmt.Sprintf("skipping %s: %v", filepath.Base(sessions[i].FilePath), err)}) + warnMu.Unlock() + } else { + profiles[i] = ProfileSession(full) + } + if tick != nil { + tick() + } + } + }() + } + for i := range sessions { + next <- i + } + close(next) + wg.Wait() + + var anchorProfile *SessionProfile + others := make([]*SessionProfile, 0, len(profiles)) + for i, p := range profiles { + if p == nil { + continue + } + if sessions[i].FilePath == anchor.FilePath { + anchorProfile = p + continue + } + others = append(others, p) + } + if anchorProfile == nil { + return nil, warnings, fmt.Errorf("could not parse anchor session %s", anchor.ID) + } + return RelateSessions(anchorProfile, others), warnings, nil +} diff --git a/internal/web/server.go b/internal/web/server.go index cec3240..57328ca 100644 --- a/internal/web/server.go +++ b/internal/web/server.go @@ -100,6 +100,7 @@ func Serve(addr string, backend provider.Backend) error { mux.HandleFunc("/api/sessions", handleAPISessions) mux.HandleFunc("/api/sessions/", handleAPISessions) mux.HandleFunc("/api/session/", handleAPISession) + mux.HandleFunc("/api/related/", handleAPIRelated) mux.HandleFunc("/api/stats", handleAPIStats) mux.HandleFunc("/api/settings", handleAPISettings) mux.HandleFunc("/api/export/", handleAPIExport) @@ -789,6 +790,54 @@ func handleAPISession(w http.ResponseWriter, r *http.Request) { _ = json.NewEncoder(w).Encode(fullSession) } +// handleAPIRelated serves the anchor session's connections to the other +// sessions of its workspace (docs/design/0006-session-connections.md): +// GET /api/related// -> ccx.related.v1, the same +// envelope as `ccx related --json`. Costs a parse of every workspace +// session (cached after the first call), so callers should fetch it on +// demand, not on every page load. +func handleAPIRelated(w http.ResponseWriter, r *http.Request) { + path := strings.TrimPrefix(r.URL.Path, "/api/related/") + parts := strings.SplitN(path, "/", 2) + if len(parts) != 2 { + http.NotFound(w, r) + return + } + projectName, sessionID := parts[0], parts[1] + session, err := sessionProvider.FindSession(projectName, sessionID) + if err != nil || session == nil { + http.NotFound(w, r) + return + } + related, warnings, err := trace.RelateWorkspace(sessionProvider, session, 4, nil) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + limit := 50 + if v := r.URL.Query().Get("limit"); v != "" { + if n, err := strconv.Atoi(v); err == nil && n >= 0 { + limit = n + } + } + total := len(related) + if limit > 0 && total > limit { + related = related[:limit] + } + if related == nil { + related = []trace.RelatedSession{} + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(map[string]any{ + "kind": "ccx.related.v1", + "session": map[string]any{"id": session.ID, "provider": session.Provider, "project": session.ProjectName, "path": session.FilePath}, + "related": related, + "total": total, + "shown": len(related), + "warnings": warnings, + }) +} + func handleAPIStats(w http.ResponseWriter, r *http.Request) { projects, err := sessionProvider.DiscoverProjects() if err != nil { diff --git a/internal/web/server_test.go b/internal/web/server_test.go index ea5c36a..6ddc170 100644 --- a/internal/web/server_test.go +++ b/internal/web/server_test.go @@ -1386,3 +1386,72 @@ func TestHandleAPIFile_DeniesSymlinkEscape(t *testing.T) { t.Fatalf("handleAPIFile returned %d, want %d. Body: %s", w.Code, http.StatusForbidden, w.Body.String()) } } + +// /api/related// returns the ccx.related.v1 envelope +// with the anchor's connections; two sessions in one workspace where +// the later one reads what the earlier one wrote relate as builds_on. +func TestHandleAPIRelated(t *testing.T) { + dir := setupTestDir(t) + projectDir := filepath.Join(dir, "projects", "-test-project") + later := `{"type":"user","timestamp":"2024-01-02T10:00:00Z","uuid":"u2","sessionId":"later-456","cwd":"/test/project","message":{"content":"continue"}} +{"type":"assistant","timestamp":"2024-01-02T10:00:01Z","uuid":"a2","parentUuid":"u2","message":{"content":[{"type":"tool_use","id":"t1","name":"Read","input":{"file_path":"/test/project/main.go"}}]}} +` + earlier := `{"type":"user","timestamp":"2024-01-01T10:00:00Z","uuid":"u1","sessionId":"test-session-123","cwd":"/test/project","message":{"content":"Hello"}} +{"type":"assistant","timestamp":"2024-01-01T10:00:01Z","uuid":"a1","parentUuid":"u1","message":{"content":[{"type":"tool_use","id":"t0","name":"Write","input":{"file_path":"/test/project/main.go","content":"x"}}]}} +` + if err := os.WriteFile(filepath.Join(projectDir, "test-session-123.jsonl"), []byte(earlier), 0644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(projectDir, "later-456.jsonl"), []byte(later), 0644); err != nil { + t.Fatal(err) + } + setTestBackend(dir) + + req := httptest.NewRequest("GET", "/api/related/-test-project/later-456", nil) + w := httptest.NewRecorder() + handleAPIRelated(w, req) + if w.Code != http.StatusOK { + t.Fatalf("status %d: %s", w.Code, w.Body.String()) + } + var resp struct { + Kind string `json:"kind"` + Total int `json:"total"` + Related []struct { + SessionID string `json:"session_id"` + Strength string `json:"strength"` + Relations []struct { + Kind string `json:"kind"` + Paths []string `json:"paths"` + } `json:"relations"` + } `json:"related"` + } + if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil { + t.Fatalf("decode: %v: %s", err, w.Body.String()) + } + if resp.Kind != "ccx.related.v1" || resp.Total != 1 || len(resp.Related) != 1 { + t.Fatalf("envelope: %+v", resp) + } + r := resp.Related[0] + if r.SessionID != "test-session-123" || r.Strength != "medium" { + t.Fatalf("related: %+v", r) + } + kinds := map[string]bool{} + for _, rel := range r.Relations { + kinds[rel.Kind] = true + } + if !kinds["builds_on"] || !kinds["previous"] { + t.Fatalf("relations: %+v", r.Relations) + } + + // Unknown session -> 404; malformed path -> 404. + w = httptest.NewRecorder() + handleAPIRelated(w, httptest.NewRequest("GET", "/api/related/-test-project/nope", nil)) + if w.Code != http.StatusNotFound { + t.Fatalf("unknown session status %d", w.Code) + } + w = httptest.NewRecorder() + handleAPIRelated(w, httptest.NewRequest("GET", "/api/related/only-one-part", nil)) + if w.Code != http.StatusNotFound { + t.Fatalf("malformed path status %d", w.Code) + } +} From a1307a3fae41ca566bd7691fb3a46274b7cf979e Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 06:07:48 -0700 Subject: [PATCH 10/13] feat(codex): turn_aborted(interrupted) is an interrupt in parser and log Codex's counterpart of Claude Code's "[Request interrupted by user]": event_msg.turn_aborted with reason interrupted (144 in one real store) now becomes a KindInterrupt marker in the full parse and an `interrupt` record in sessionlog, so trace interrupt counts and `log --kind interrupt` cover both providers. Other abort reasons are left alone. Tests: TestParseSessionCodexTurnAbortedIsInterrupt; legacy rollout log test extended. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 1 + internal/provider/codex/backend.go | 26 ++++++++++++ internal/provider/codex/interrupt_test.go | 48 +++++++++++++++++++++++ internal/sessionlog/sessionlog.go | 8 ++++ internal/sessionlog/sessionlog_test.go | 5 +++ 5 files changed, 88 insertions(+) create mode 100644 internal/provider/codex/interrupt_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 50e819c..bc06531 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) - **`ccx search --content` reports when: `FIRST` column, `first_hit` in `--json`, `--sort first|last|hits`.** "When did we first mention X" needs the earliest matching message and an oldest-first order; results only carried session end time and ranked by hit count. Each content hit now records the timestamp of its earliest matching message (parsed messages by default; the raw line's top-level `timestamp` under `--raw`), printed as `FIRST` and sortable with `--sort first`; `--sort last` orders by session activity; `--sort hits` is the old order and the default. ### Added +- **Codex interruptions count too.** `event_msg.turn_aborted` with reason `interrupted` (144 in one store) is Codex's "[Request interrupted by user]"; it now parses as the same `interrupt` kind, so `trace` interrupt counts and `log --kind interrupt` cover both providers. - **Human interventions are first-class in `trace` and `log`.** "[Request interrupted by user]" (the human pressed stop) and permission-prompt rejections ("The user doesn't want to proceed with this tool use") are the human in the loop, but ccx read the first as a *prompt* — it opened a fake turn `u: [Request interrupted by user for tool use]` — and the second as an ordinary tool error. New parser kind `interrupt` (harness marker, never an exchange anchor) and `parser.IsToolDenial`; `trace` counts `interrupts` and `denials` on turn, step, and stats, marks the rejected call `denied` (not an error, it did not run), and badges them in the outline header and rows (`1 interrupt, 2 denied`; step `[4t 1! 1d]`); `log` reports kinds `interrupt` and `tool_denied` (`ccx log --scope today --all --kind interrupt,tool_denied`). 650 interruptions across 321 sessions and 145 rejections in one real store were invisible before. - **`GET /api/related//` serves session connections as JSON** — the `ccx.related.v1` envelope (`related`, `total`, `shown`, `warnings`; `?limit=N`), computed by the same `trace.RelateWorkspace` the CLI uses, so a web panel or an agent reading the API sees exactly what `ccx related --json` prints. Fetched on demand: it costs a parse of every workspace session (cached after the first call). diff --git a/internal/provider/codex/backend.go b/internal/provider/codex/backend.go index 3dce30e..340eb8c 100644 --- a/internal/provider/codex/backend.go +++ b/internal/provider/codex/backend.go @@ -81,6 +81,12 @@ type agentMessagePayload struct { Message string `json:"message"` } +type turnAbortedPayload struct { + Type string `json:"type"` + TurnID string `json:"turn_id"` + Reason string `json:"reason"` +} + type agentReasoningPayload struct { Text string `json:"text"` } @@ -1004,6 +1010,26 @@ func (b *Backend) parseSession(filePath string, threadNames map[string]string) ( )) } + case "turn_aborted": + // The human stopped the turn (Esc / Ctrl-C). Codex's + // counterpart of Claude Code's "[Request interrupted + // by user]": a harness marker, never an exchange + // anchor, counted by trace as an interrupt. + var payload turnAbortedPayload + if err := json.Unmarshal(rollout.Payload, &payload); err != nil { + continue + } + if payload.Reason == "" || payload.Reason == "interrupted" { + messages = append(messages, newMessage( + fmt.Sprintf("codex-interrupt-%d", lineNum), + "user", + parser.KindInterrupt, + ts, + currentModel, + parser.ContentBlock{Type: "text", Text: "[Turn interrupted by user]"}, + )) + } + case "agent_reasoning", "agent_reasoning_raw_content": var payload agentReasoningPayload if err := json.Unmarshal(rollout.Payload, &payload); err != nil { diff --git a/internal/provider/codex/interrupt_test.go b/internal/provider/codex/interrupt_test.go new file mode 100644 index 0000000..4fb70bc --- /dev/null +++ b/internal/provider/codex/interrupt_test.go @@ -0,0 +1,48 @@ +package codex + +import ( + "path/filepath" + "testing" + + "github.com/thevibeworks/ccx/internal/parser" +) + +// event_msg.turn_aborted (reason interrupted) is the human stopping a +// Codex turn — Codex's "[Request interrupted by user]". It must parse +// as a KindInterrupt marker (never a prompt) so trace counts it, and +// other abort reasons must not. +func TestParseSessionCodexTurnAbortedIsInterrupt(t *testing.T) { + home := t.TempDir() + sessionsDir := filepath.Join(home, "sessions") + rolloutPath := filepath.Join(sessionsDir, "2026", "08", "18", "rollout-abort.jsonl") + writeRollout(t, rolloutPath, `{"timestamp":"2026-08-18T10:00:00Z","type":"session_meta","payload":{"id":"abort-1","cwd":"/tmp/repo","timestamp":"2026-08-18T10:00:00Z"}} +{"timestamp":"2026-08-18T10:00:01Z","type":"event_msg","payload":{"type":"user_message","message":"run the migration"}} +{"timestamp":"2026-08-18T10:00:02Z","type":"event_msg","payload":{"type":"agent_message","message":"Starting."}} +{"timestamp":"2026-08-18T10:00:03Z","type":"event_msg","payload":{"type":"turn_aborted","turn_id":"t1","reason":"interrupted"}} +{"timestamp":"2026-08-18T10:00:04Z","type":"event_msg","payload":{"type":"turn_aborted","turn_id":"t2","reason":"replaced"}} +`) + backend := NewWithDirs(home, sessionsDir, filepath.Join(home, "archived_sessions")) + session, err := backend.ParseSession(rolloutPath) + if err != nil { + t.Fatalf("ParseSession: %v", err) + } + interrupts := 0 + prompts := 0 + for _, m := range parser.FlattenSessionMessages(session) { + switch m.Kind { + case parser.KindInterrupt: + interrupts++ + if m.Type != "user" || m.Content[0].Text != "[Turn interrupted by user]" { + t.Fatalf("interrupt message shape: %+v", m) + } + case parser.KindUserPrompt: + prompts++ + } + } + if interrupts != 1 { + t.Fatalf("interrupts = %d, want 1 (only reason=interrupted counts)", interrupts) + } + if prompts != 1 { + t.Fatalf("prompts = %d, want 1 (the abort marker is not a prompt)", prompts) + } +} diff --git a/internal/sessionlog/sessionlog.go b/internal/sessionlog/sessionlog.go index ec43fb4..5565afd 100644 --- a/internal/sessionlog/sessionlog.go +++ b/internal/sessionlog/sessionlog.go @@ -770,6 +770,14 @@ func normalizeCodexEvent(record *Record, payload map[string]any, payloadType str case "compacted": record.Kind = "compaction" record.Text = truncateText(cleanText(stringField(payload, "message")), 1000) + case "turn_aborted": + // The human stopped the turn: same kind as Claude's + // "[Request interrupted by user]" so --kind interrupt + // covers both providers. + record.Kind = "interrupt" + record.Role = "user" + record.TurnID = stringField(payload, "turn_id") + record.Text = truncateText(cleanText(joinNonEmpty(" ", "turn aborted:", emptyDefault(stringField(payload, "reason"), "interrupted"))), 1000) default: record.Kind = emptyDefault(payloadType, "event") record.Text = truncateText(cleanText(contentPreview(payload)), 1000) diff --git a/internal/sessionlog/sessionlog_test.go b/internal/sessionlog/sessionlog_test.go index 22e50da..9f3433a 100644 --- a/internal/sessionlog/sessionlog_test.go +++ b/internal/sessionlog/sessionlog_test.go @@ -229,6 +229,7 @@ func TestCollectCodexLegacyRolloutUnchanged(t *testing.T) { `{"timestamp":"2026-05-21T00:01:00Z","type":"session_meta","payload":{"id":"codex-legacy","cwd":"/tmp/repo"}}`, `{"timestamp":"2026-05-21T00:02:00Z","type":"event_msg","payload":{"type":"user_message","message":"legacy prompt"}}`, `{"timestamp":"2026-05-21T00:03:00Z","type":"event_msg","payload":{"type":"agent_message","message":"legacy reply"}}`, + `{"timestamp":"2026-05-21T00:04:00Z","type":"event_msg","payload":{"type":"turn_aborted","turn_id":"t1","reason":"interrupted"}}`, ) bundle, err := Collect([]Source{{Provider: "codex", Home: home}}, Options{Start: mustParseTime(t, "2026-05-21T00:00:00Z"), End: mustParseTime(t, "2026-05-22T00:00:00Z")}) if err != nil { @@ -237,6 +238,10 @@ func TestCollectCodexLegacyRolloutUnchanged(t *testing.T) { if bundle.Metrics.UserPrompts != 1 || bundle.Metrics.AssistantMessages != 1 { t.Fatalf("legacy rollout must keep counting: %+v", bundle.Metrics) } + last := bundle.Records[len(bundle.Records)-1] + if last.Kind != "interrupt" || last.Role != "user" || last.TurnID != "t1" { + t.Fatalf("turn_aborted must be an interrupt record: %+v", last) + } } // user-role lines that no human typed — command markers, local command From 1b1c2bd64a61f4f7f69db08b5d89275c13aa2aa9 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Tue, 18 Aug 2026 06:08:30 -0700 Subject: [PATCH 11/13] docs(devlog): session connections, human interventions, citation walk Co-Authored-By: Claude Fable 5 --- ...6-08-18-session-connections-human-loop.org | 90 +++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 docs/devlog/2026-08-18-session-connections-human-loop.org diff --git a/docs/devlog/2026-08-18-session-connections-human-loop.org b/docs/devlog/2026-08-18-session-connections-human-loop.org new file mode 100644 index 0000000..c98c94d --- /dev/null +++ b/docs/devlog/2026-08-18-session-connections-human-loop.org @@ -0,0 +1,90 @@ +* [2026-08-18] Dev Log: Session connections, human interventions, citations walk :TRACE:LOG:SEARCH:RELATED: + +** Context +Eric's brief: "continue to think and improve ccx; inspired by +MineContext" with the core values spelled out — session trajectory, +how decisions were made, traceable tool calls, long context folded +short, the whole timeline and the humans in the loop, all agent +context, and the connections between sessions. Autonomous run, +commit-and-continue. Two references studied: semantica-agi/semantica +(design 0005: every fact carries a quote + location; supersedes vs +derives-from; bands not decimals) and volcengine/MineContext (capture +-> process -> consume; for ccx capture is free, the missing layer was +*connect*). + +** Why +Against the value list, the store answered "what happened in this +session" well and "how sessions relate", "where did the human step +in", and "walk me to that citation" badly or not at all: +- sessions were islands: forks, handoffs, parallel agents, and + "see 736a7bac" mentions were in the transcripts, uncomputed; +- "[Request interrupted by user]" parsed as a prompt and opened a fake + turn; permission rejections counted as tool errors; Codex + turn_aborted was invisible — 650 + 145 + 144 human interventions in + one real store, none named; +- =ccx log= reported 233 "prompts" for one day, 79 typed by a human, + and rendered Codex 0.147 conversations as bare =item_completed=; +- search/trace handed out message ids nothing could open; +- prompts older than the oldest session file were unreachable. + +** What +- =FEAT= =ccx related [session]= + =related= in =trace --full= + + =GET /api/related//= (design 0006): forked_from / + fork_of (shared message uuids), mentions / mentioned_by (id prefix in + conversation text, quoted), handoff_from / handoff_to (baton file + written by one, read by the other later), builds_on / built_on_by, + overlaps, previous / next. Strength bands; capped path lists with + count + truncated; evidence = session, message id, time, path, + quote. =trace.RelateWorkspace= shared by CLI and web. +- =FEAT= human interventions: parser =KindInterrupt= (Claude + "[Request interrupted by user…]", Codex =turn_aborted(interrupted)=) + and =IsToolDenial=; trace counts interrupts/denials on turn, step, + stats; denied calls marked =denied= (not errors); outline header and + badges; log kinds =interrupt= / =tool_denied=. +- =FEAT= =ccx log --kind K1,K2= and =--match PHRASE [-w]= — the firehose + becomes a timeline: "the humans in the loop, today" is + =ccx log --scope today --all --kind user_prompt=. +- =FEAT= =ccx view --at [--context N]= — walk from + a citation to its context (open since 08-03 finding 4). +- =FEAT= prompt history (=~/.claude/history.jsonl=, =~/.codex/history.jsonl=) + as a =search --content= source, type =prompt=, only for prompts whose + session file is gone. +- =FIX= =ccx log= applies the parser's rules: Claude command markers / + echoes / notifications / meta / compaction are not prompts; Codex + 0.147 item_completed rows are the conversation, raw response_item + and duplicated legacy events demoted (=model_input= / + =model_output= / =legacy_message=); metrics count once; tool_result + rows preview content. +- =FIX= heredoc bodies were scanned as shell redirects (=if n > 0= + inside =python3 - <<'EOF'= became an "edited file"). +- =CHANGE= narration-less trace steps get "(no narration) Bash x3, + Read" instead of a bare badge row (08-17 finding 3). + +** How +Commits, in order: 8819774 related · afa3785 log kinds + filters · +3a8644f interventions + step labels · 8d8f77a view --at · bb9f07f +history source · c0d07a5 skills/roadmap · 0f79cba /api/related + +RelateWorkspace · a1307a3 codex interrupts. Each with symptom-named +tests; =go test ./...= green throughout. + +** Decisions +| Decision | Alternatives | Rationale | DRI | Timestamp | +| relations recomputed from transcripts, no graph store | SQLite edge table | the transcripts are the store; ccx is read-only over them (0005 rejected list) | Fable 5 | 2026-08-18 | +| strength = bands, never floats | similarity scores | 0005 principle 7; a fork or a handoff is strong by kind, not by arithmetic | Fable 5 | 2026-08-18 | +| history prompts only when the session is gone | always list | one prompt, one citation; the session file is the better anchor while it exists | Fable 5 | 2026-08-18 | +| interrupt is a Kind, not a flag on user_prompt | flag | it must never anchor a turn; renderers' default branches handle it | Fable 5 | 2026-08-18 | +| /api/related endpoint only, no web panel yet | panel now | no display here to verify a UI change against the 0002 shell contract | Fable 5 | 2026-08-18 | + +** Dogfood findings (open) +1. =--match -w X= is a footgun: cobra takes =-w= as the value of + =--match=. Documented as =--match X -w=; a =--match= that rejects + values starting with =-= would be kinder. +2. Web session page does not yet show =related= or interrupt/denial + badges; the API and trace fields are there. Needs a visual pass. +3. Codex approval rejections (the counterpart of Claude's denial) were + not found as a distinct event in the store; only interrupts are + covered for Codex. +4. =related= scans every session of the workspace; a workspace with + thousands of sessions will feel it on a cold cache (progress shows, + parse cache makes the second run ~1s). A time window flag would + bound it if that bites. From 9970a18ccefbec74d36a654beff35dfa409ad1ad Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Wed, 19 Aug 2026 20:03:20 -0700 Subject: [PATCH 12/13] fix(search): --hits citations print the whole message id MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The MESSAGE column cut every id to 8 characters, which is fine for a uuid and wrong for the synthetic ids: `codex-thinking-90` and `codex-thinking-331` both printed as `codex-th`, and a prompt-history anchor at line 4426 printed as `line:442` — a different line that happens to exist. So the citation was not merely unpasteable into `ccx view --at` ("prefix is ambiguous"); it pointed somewhere else. Shorten only ids whose first 8 characters are hex; print the rest whole. They are short to begin with, so the column stays narrow. --- internal/cmd/search.go | 25 ++++++++++++++++++++++++- internal/cmd/search_test.go | 23 +++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-) diff --git a/internal/cmd/search.go b/internal/cmd/search.go index 16a9409..934a232 100644 --- a/internal/cmd/search.go +++ b/internal/cmd/search.go @@ -547,6 +547,29 @@ func truncateID(id string, max int) string { return id[:max] } +// citationID renders a message id for the MESSAGE column, which is +// the value a reader pastes into `ccx view --at`. A uuid stays +// unambiguous at its 8-hex prefix, so shorten it; a synthetic id +// (codex-thinking-13, line:442) is short already and every one of +// them collides at 8 chars, so print it whole or the citation walk +// dead-ends on "prefix is ambiguous". +func citationID(id string) string { + if len(id) > 8 && isHexRun(id[:8]) { + return id[:8] + } + return id +} + +func isHexRun(s string) bool { + for i := 0; i < len(s); i++ { + c := s[i] + if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')) { + return false + } + } + return len(s) > 0 +} + // sessionParser is the slice of provider.Backend the conversation // scan needs; narrowed so tests can stub it. type sessionParser interface { @@ -762,7 +785,7 @@ func printSearchHits(hits []searchHit) error { if session == "" { session = "-" } - fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", t, session, h.Role, truncateID(id, 8), cleanDisplayText(h.Quote)) + fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", t, session, h.Role, citationID(id), cleanDisplayText(h.Quote)) } return w.Flush() } diff --git a/internal/cmd/search_test.go b/internal/cmd/search_test.go index 04167e2..30222e4 100644 --- a/internal/cmd/search_test.go +++ b/internal/cmd/search_test.go @@ -544,3 +544,26 @@ func TestScanPromptHistory(t *testing.T) { t.Fatalf("prompt result: %+v", r) } } + +// A citation is only useful if it can be pasted back: `ccx search +// --hits` prints MESSAGE, `ccx view --at` consumes it. Uuids shorten +// to 8 hex; synthetic ids (codex-thinking-13, line:442) must survive +// whole, since every one of them is ambiguous at 8 characters. +func TestCitationIDStaysPasteable(t *testing.T) { + cases := []struct { + id string + want string + }{ + {"cf332028-7a1e-4bd9-9a2f-9d1c2b3a4f55", "cf332028"}, + {"codex-thinking-13", "codex-thinking-13"}, + {"codex-thinking-129", "codex-thinking-129"}, + {"line:442", "line:442"}, + {"", ""}, + {"abc123", "abc123"}, + } + for _, c := range cases { + if got := citationID(c.id); got != c.want { + t.Errorf("citationID(%q) = %q, want %q", c.id, got, c.want) + } + } +} From 245a0a3b0eabfb80fbe7c400f5aff3048d150e60 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Wed, 19 Aug 2026 20:03:53 -0700 Subject: [PATCH 13/13] docs(changelog): note the citation-id fix; one Added section in Unreleased --- CHANGELOG.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bc06531..64dc695 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,11 +12,8 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) - **`ccx search -w/--word` matches whole words.** Matching was substring-only, so a term that prefixes a common word was unanswerable: `search --content semantica` returned 47 sessions, 46 of them "semantic*ally*", and ccx alone could not tell 0 real hits from 46 (docs/devlog/2026-08-18-search-word-boundary-dogfood.org). `-w` demands an ASCII word boundary on each side of the query that starts/ends with a word character (so "semantica-agi" and "(semantica)" still hit; CJK queries are unaffected) and applies to names, summaries, conversation text, and `--raw` lines alike. - **`ccx search --hits` turns matches into citations.** One row per matching message — time, session, role, message id, quote — oldest first across sessions, `-n`-capped with a visible "showing N of M". The anchors are the same ones `trace` and `view` use, so a claim built on a search can point at its evidence (design: docs/design/0005-evidence-citations-lessons-from-semantica.md). Under `--raw` the unit is a transcript line, anchored by its own `uuid`/`type`/`timestamp`. - **`ccx search --content` reports when: `FIRST` column, `first_hit` in `--json`, `--sort first|last|hits`.** "When did we first mention X" needs the earliest matching message and an oldest-first order; results only carried session end time and ranked by hit count. Each content hit now records the timestamp of its earliest matching message (parsed messages by default; the raw line's top-level `timestamp` under `--raw`), printed as `FIRST` and sortable with `--sort first`; `--sort last` orders by session activity; `--sort hits` is the old order and the default. - -### Added - **Codex interruptions count too.** `event_msg.turn_aborted` with reason `interrupted` (144 in one store) is Codex's "[Request interrupted by user]"; it now parses as the same `interrupt` kind, so `trace` interrupt counts and `log --kind interrupt` cover both providers. - **Human interventions are first-class in `trace` and `log`.** "[Request interrupted by user]" (the human pressed stop) and permission-prompt rejections ("The user doesn't want to proceed with this tool use") are the human in the loop, but ccx read the first as a *prompt* — it opened a fake turn `u: [Request interrupted by user for tool use]` — and the second as an ordinary tool error. New parser kind `interrupt` (harness marker, never an exchange anchor) and `parser.IsToolDenial`; `trace` counts `interrupts` and `denials` on turn, step, and stats, marks the rejected call `denied` (not an error, it did not run), and badges them in the outline header and rows (`1 interrupt, 2 denied`; step `[4t 1! 1d]`); `log` reports kinds `interrupt` and `tool_denied` (`ccx log --scope today --all --kind interrupt,tool_denied`). 650 interruptions across 321 sessions and 145 rejections in one real store were invisible before. - - **`GET /api/related//` serves session connections as JSON** — the `ccx.related.v1` envelope (`related`, `total`, `shown`, `warnings`; `?limit=N`), computed by the same `trace.RelateWorkspace` the CLI uses, so a web panel or an agent reading the API sees exactly what `ccx related --json` prints. Fetched on demand: it costs a parse of every workspace session (cached after the first call). - **`ccx search --content` scans prompt history.** Claude Code and Codex append every human prompt to `history.jsonl`, and those files outlive session cleanup (one real store: prompts back to 2025-09-28, sessions back to 2025-12-07). Matches surface as type `prompt` — only for prompts whose session file is no longer in the store, so a prompt never appears twice — with `FIRST`, a `[user]` quote, and under `--hits` a `history:line` anchor. "When did we first say X" now reaches past the session horizon. - **`ccx view --at MESSAGE_ID [--context N]` walks from a citation to its context.** Search `--hits` and `trace` hand out message ids, but nothing in the CLI could open one; drill-down meant the web page or raw grep (open since docs/devlog/2026-08-03-content-search-noise.org finding 4). `--at` renders the cited message with N messages before and after it (wire order, flattened; the target survives `--brief`), and says where it sits: `message 1 of 763`. Prefixes resolve; ambiguous prefixes are an error, not a guess. @@ -33,6 +30,7 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) - **`-n` is the `--limit` shorthand everywhere.** Only `search` had it; `sessions -n 2` failed with "unknown shorthand flag". `sessions`, `projects`, and `log` now accept `-n` too. - **A session whose summary matched dropped its content evidence.** The summary hit short-circuited the content scan, so under `--content` the session where the term was actually discussed could be the one result with no hit count, previews, or first-hit time. Summary hits stay typed `session` but now carry the content fields. - **Codex 0.147 conversations render again.** Codex moved its UI-facing user and assistant records from legacy `event_msg.user_message` / `agent_message` events to canonical `event_msg.item_completed` TurnItems. The Codex adapter now selects one conversation source per rollout, reads stable `UserMessage` / `AgentMessage` item IDs and content, keeps legacy rollouts working, and never mistakes raw `response_item` model input (which can contain injected instruction envelopes) for a human prompt. Discovery metadata, terminal view, web, export, search, and trace now agree; the parse-cache format is bumped so upgrades cannot serve blank cached sessions. +- **`search --hits` citations print the whole message id.** The MESSAGE column cut every id to 8 characters. That is unambiguous for a uuid and wrong for the synthetic ids: `codex-thinking-90` and `codex-thinking-331` both printed as `codex-th`, and a prompt-history anchor at line 4426 printed as `line:442` — a line that exists, and is not the one cited. The citation was not merely unpasteable into `ccx view --at`; it pointed elsewhere. Only hex-prefixed ids are shortened now. ## [0.15.0] - 2026-08-11