Skip to content

feat(chat): derive readable summaries for purpose-less shell labels - #6435

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
jingchaodev:feat/shell-label-summaries
Aug 28, 2026
Merged

feat(chat): derive readable summaries for purpose-less shell labels#6435
bolichen97 merged 1 commit into
kirodotdev:mainfrom
jingchaodev:feat/shell-label-summaries

Conversation

@jingchaodev

@jingchaodev jingchaodev commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

A tool call that carries no purpose renders its raw title as the pill label (pickToolLabel fallback), and a shell title is the command verbatim. #6152 pins the collapsed label to one line, which bounds how much of the command is visible — but the visible part of a clipped command is a wall of shell quoting that says almost nothing: cat > /tmp/desc.md <<'EOF' ### Notes Two changes that….

This is the majority case for shell, not an edge case: in one real session, 328 of 442 tool messages carried no purpose — including 126 of 126 shell calls, all persisted as bare titles without the Running: prefix.

Why it matters

Anyone reading a transcript in simplified mode ("Simplified Tool Call Names" ON — the default) scans pill labels to follow what the agent did. For purpose-less shell calls, today's label is clipped raw code, so the reader must click every pill open to learn even which programs ran. Purposes are optional model output even where the tool schema declares the field, so this fallback path is permanent, not transitional — it needs to read well on its own.

What changed (motivation → approach → change)

Goal: make the one visible line meaningful for purpose-less shell calls.

Approach: a deterministic, render-time command digest, chosen over two alternatives — a render-time LLM summarizer (per-view token cost, async flicker, non-reproducible labels, and a prompt-injection surface) and a string clamp of the raw text (built first, then dropped: #6152's CSS truncate landed on main and already bounds visibility, and a string clamp would fight #6152's deliberate wrap-restore on expand). Substituting content composes cleanly with #6152's visibility bound.

Change: when Simplified Tool Call Names is ON and a purpose-less shell label is flood-length (> DERIVE_LABEL_THRESHOLD_CHARS or multi-line), the pill substitutes a digest via deriveShellSummary (new, website/src/utils/toolLabel.ts):

  • Running: cat > /tmp/desc.md <<'EOF' + 30-line body → Running: cat → /tmp/desc.md
  • Running: export PATH=… cd …; ls -a | grep -i crux || echo none; git …; wc …Running: ls, grep, echo, git …

Mechanics: quote-masked segment splitting (grep -E 'foo|bar' stays one command), redirect masking (2>&1 contributes nothing), env-assignment skipping, bookkeeping demotion (export/cd dropped when a meaningful binary exists, kept when alone), heredoc bodies excluded, binaries deduped and capped at 4, first redirect target shown as → target. Bare titles derive too, gated on the tool log's is_shell so a non-shell title (Editing AGENTS.md) can never be misparsed; unparseable labels return null and keep the raw text under #6152's truncate. The verbatim command moves to the row's title (hover) whenever a substitute is shown and remains unchanged in the expanded panel. Short labels (Running: git status) render byte-identically in every mode; raw mode (toggle OFF) always shows the exact command. Output is command names and paths only — script-neutral, no i18n catalog entry, zero model involvement. Labels are computed at render time, so existing transcripts improve on next view.

Tests

  • 11 unit tests for deriveShellSummary (new src/test/toolLabel.test.ts): heredoc → binary + redirect target; pipeline binaries deduped/capped with bookkeeping demotion; lone-builtin retention; multi-line scripts read past a bookkeeping first line; heredoc bodies contribute no binaries; quoted operators not split; 2>&1 not a segment/target; env-assignment skipping; MCP/non-shell titles return null; bare-command gating on the caller's vouch; pickToolLabel fallback interplay.
  • 2 component cases in ToolCallLine.test.tsx: flood-length purpose-less shell call renders the derived digest (and none of the heredoc body); short shell label passes through untouched in simplified mode.
  • fix(chat): fold a long tool-call title to one line while collapsed #6152's ToolCallLine.labelClamp.test.tsx passes unmodified alongside the substitution.
  • tsc -b clean; eslint 0 errors on changed files.

Manual verification

Verified in an isolated dev gateway (worktree source, own data home): a purpose-omitted heredoc call rendered the derived digest with the full command on hover and verbatim in the expanded panel; a purpose-carrying call still showed its purpose; Running: git status unchanged; toggle OFF showed the exact raw command. Hover title is the one behavior jsdom cannot assert, hence the manual pass.

Screenshots / video

All captured on a live gateway running this branch, Simplified Tool Call Names ON, purposes deliberately omitted so the fallback path renders.

Redirect-target form — the original bug's shape (multi-line heredoc write; pill reads cat → /tmp/screenshot-demo.md, diff card below is main's #5012 feature coexisting):

derived redirect target

Pipeline form (208-char chained command; pill reads echo, tr, wc, git):

derived pipeline

No-regression (short command, still purpose-less; pill stays verbatim git status):

short label passthrough

Baseline for comparison: with the toggle OFF (or on current main), the same calls render the raw command truncated to one line by #6152 — content identical to the title/expanded-panel text visible above.

Related Issues

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable) — N/A: rendering-only change, no documented behavior contract touched
  • No secrets, credentials, or internal references in the diff

@jingchaodev
jingchaodev requested a review from a team August 27, 2026 23:56
@jingchaodev
jingchaodev requested a review from a team as a code owner August 27, 2026 23:56
@jingchaodev
jingchaodev requested a review from bolichen97 August 27, 2026 23:56
@dwu96

dwu96 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated.

Missing sections:

  • ## Problem / Motivation
  • ## Why it matters
  • ## What changed
  • ## Tests

Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle.

1 similar comment
@iamwhatever

Copy link
Copy Markdown
Collaborator

👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated.

Missing sections:

  • ## Problem / Motivation
  • ## Why it matters
  • ## What changed
  • ## Tests

Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle.

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Aug 28, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Aug 28, 2026
@jingchaodev

Copy link
Copy Markdown
Contributor Author

Check triage: the De-Amazon Scrub Lint hit was mine (an internal build-tool name in a comment and test fixture) — removed in the follow-up commit. The Frontend Lint & Type Check failure is jscpd flagging a 14-line clone between website/scripts/capture-chatpane-queue-edit.mjs and capture-chatpane-upload-error.mjs — both from #5711 on main, untouched by this PR; it should reproduce on any branch cut from current main. Screenshot Evidence is legitimate and a screenshot is being prepared.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Aug 28, 2026
@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed edebe4027993300da87b7c2ae466f843442fd8c1 via the fork AI-review pipeline; updated in place on each push.

Review details

FINDING -- website/src/pages/chat/ToolCallLine.tsx:670 -- "bareCommand: isShell" skips reloaded kind: "execute" commands and rewrites multiline purposes -> Fix: require toolLabel === label and include toolKind === 'execute' in the shell gate.
[GPT-REVIEWED] edebe40

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — 🟡 CONCERNS

Design-level review of edebe4027993300da87b7c2ae466f843442fd8c1 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The gate implements "long label," not the stated "purpose-less label" — a long model-authored purpose on a shell call gets mangled into shell tokens.

Watch

  • pillLabelText substitutes whenever displayLabel exceeds the threshold, but displayLabel is the purpose when one exists (pickToolLabel), and bareCommand: isShell vouches it as shell. So a >200-char (or multi-line) purpose on a shell call — "Run the full test suite, then…" — is parsed as a command and collapses to its first prose word ("Run"), replacing a meaningful label with garbage. The PR's own scope sentence ("a purpose-less shell label is flood-length") names the missing condition; purpose is already in scope in the component — add purpose-absence to the gate.

Suggestions

  • The PR adds src/test/toolLabel.test.ts while the base already has a colocated src/utils/toolLabel.test.ts for the same module — put the new cases in the existing file so the module's tests don't split across two locations.

[DESIGN-REVIEWED] edebe40

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

UX Review (Fable 5, fork) — 🟡 CONCERNS

UX-level review of edebe4027993300da87b7c2ae466f843442fd8c1 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

I have what I need. The decision surfaces (approval bar, ghost pill, locked-open details panel) keep the verbatim command, so the digest never gates an approval. The remaining findings are about digest quality on specific command shapes and cross-surface parity.

UX-Verdict: CONCERNS

Wrapper-shaped commands digest to a bare wrapper name — "Running: bash" — strictly less informative than the truncated raw label it replaces.

Watch

  • Wrapper digest regression: deriveShellSummary collects only segment heads and maskQuotes blanks quoted bodies, so a 200+-char bash -c "npm run build && …", sudo …, xargs …, or timeout … label digests to just Running: bash / Running: sudo — where the old CSS-truncated raw at least showed the informative opening of the real command. Moderate impact × persistent for every such call in simplified mode; frequency unverified but plausible for agent-written shells. Smallest fix: skip known wrappers the way ENV_ASSIGN_RE prefixes are skipped, or return null (keep raw) when the digest yields only a wrapper.

Suggestions

  • toolStatusLabel.ts documents that the session-list row "agrees with the inline tool pill", but it still shows the raw flood-length wall the pill now digests — route its Running:-prefixed shell labels through deriveShellSummary for parity.

[UX-REVIEWED] edebe40

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — 🟡 CONCERNS

Premise-level review of edebe4027993300da87b7c2ae466f843442fd8c1 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

I've completed the review: read the contract, the intent file, the authoritative patch, and verified against the base tree — isShell already exists in ToolCallLine.tsx, no existing frontend summarizer (grepped summari[sz]e|digest in website/src/utils — only unrelated hits), the backend split_command_segments lives at the authorization boundary in Python (trust_patterns.py:180, not importable in the browser), temp-screenshots/ is a documented committed-deliverables convention (2,558 tracked files, README forbids asking authors to remove them), and pickToolLabel has 3 consumers of which only the pill is fixed.

First-Principles-Verdict: CONCERNS

The digest fixes the transcript pill only; the same purpose-less raw shell command still renders in the session-row status — an unfixed, unnamed sibling.

What this change ships

Intent: make the one visible line of a purpose-less shell tool pill say what the command ran. ADDITION, framed honestly as feat.

  1. Flood-length purpose-less shell pills show a derived command digest in simplified mode — justified
  2. Hover on a substituted pill reveals the verbatim command — justified
  3. Short labels and raw mode render byte-identically — justified
  4. New deriveShellSummary + exported threshold constant — justified; one consumer, but it is the mechanism
  5. Unparseable/non-shell labels return null and keep raw text — justified fail-open
  6. Screenshots committed under temp-screenshots/ — justified by documented convention (temp-screenshots/README.md)

Watch

  • Point patch with 1 counted unfixed sibling. Grepped pickToolLabel: 3 consumers — the pill (fixed here), the approval bar (ChatInput.tsx:888 — correctly exempt: an approval must show the exact command), and the session-row status (toolStatusLabel.ts:36), which falls back to the same raw toolName for purpose-less shell calls and shows the same clipped quoting the description calls the majority case. The fix is a one-line reuse of deriveShellSummary; accepted-and-deferred is fine, but the PR never names it.
  • Provenance of the harm rests on the author's own session count ("126 of 126 shell calls"), unverifiable from the base tree — plausible, but a human should know the evidence is self-reported.

Subtractions

  • Fold the new website/src/test/toolLabel.test.ts into the existing website/src/utils/toolLabel.test.ts — one module now has two test homes that will diverge.

[FIRST-PRINCIPLES-REVIEWED] edebe40

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Opus 4.8 Review (fork) — ✅ no blocking findings

Reviewed edebe4027993300da87b7c2ae466f843442fd8c1 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings block the merge.

FINDING — website/src/pages/chat/ToolCallLine.tsx:41 — a long agent purpose on a shell call (isShell true, so bareCommand: isShell is true) survives pickToolLabel as displayLabel, and once it exceeds DERIVE_LABEL_THRESHOLD_CHARS it is fed to deriveShellSummary(displayLabel, { bareCommand: isShell }) which parses the prose as a command and collapses it to its first word — so a 200+ char purpose renders as e.g. "Search" instead of the truncated purpose shown before this change → Fix: skip derivation when the label is the agent purpose rather than the raw command (thread a usedPurpose flag from pickToolLabel and only derive when it is false).

[OPUS-REVIEWED] edebe40

A tool call with no purpose falls back to its raw title, and a shell
title is the command verbatim. kirodotdev#6152 pins the collapsed label to one
line, which bounds how much is visible; this bounds what the visible
part means. Observed in a real session: 126 of 126 shell calls carried
no purpose, all as bare titles without the Running: prefix.

In simplified mode, a purpose-less flood-length shell label is
substituted with a deterministic command digest: quote-masked segment
splitting, redirect masking, bookkeeping demotion, heredoc bodies
excluded, binaries capped at 4, first redirect target as an arrow
suffix. Bare titles derive too, gated on the tool log's is_shell.
Verbatim command on hover and in the expanded panel; short labels
render unchanged. Includes screenshot evidence under temp-screenshots.
@jingchaodev
jingchaodev force-pushed the feat/shell-label-summaries branch from 4c82731 to edebe40 Compare August 28, 2026 02:38
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Aug 28, 2026
@bolichen97
bolichen97 enabled auto-merge (squash) August 28, 2026 09:12
@bolichen97
bolichen97 merged commit 2dc7cd2 into kirodotdev:main Aug 28, 2026
69 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Aug 28, 2026
@bolichen97

Copy link
Copy Markdown
Collaborator

Open PR relationship audit

This is a consolidated, point-in-time code-level audit note. It compares complete merge-base diffs and current/merged code; it does not treat a shared topic as duplication or partial coverage as completion.

Relationship findings

  • PR #8294 is PARTIALLY_COVERED relative to this PR. Coverage is explicitly incomplete; this finding is not a completion or closure claim. Recommended action for PR #8294: KEEP. PR #6435 is the merged foundation this PR builds on, not a substitute for it: the merged code covers the flood-length-title case only, and none of the primary's three behaviors (command-sourced label, interpreter/wrapper/varref stepping with subcommand attachment, historical shell-ness) exist in current origin/main. Files: website/src/pages/chat/ToolCallLine.tsx, website/src/utils/toolLabel.ts.

No PR, Issue, label, branch, or review state was changed by the relationship-note portion of this audit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants