Skip to content

Widget UX triage: 21 root-caused defects from one live session (umbrella) #589

Description

@omridevk

One live session ("Explain this page") surfaced 21 defects + 1 promoted candidate. Seven read-only RCA agents traced each to code; fix specs were adversarially reviewed (codex gpt-5.6-sol) and revised. This is the umbrella; fixes land as per-wave PRs referencing this issue.

RCA line numbers are from main @ 2991888.

Cross-cutting causes

  1. Replay offset — every subscribe yields a full snapshot then re-streams live runs from offset '-1' (T1, T11, aggravates T13).
  2. Gate policy inversionask-on-metacharacters outranks the read-only allow-list; no session memory (T2).
  3. Rail has no row identity — DOM-ordinal anchors, overloaded "active" meaning, off-by-one segment mapping (T4, T5, T12).
  4. Containers decided apart from content — frames render before knowing the body is empty; symmetric padding around asymmetric children; mixed-font flex centering (T3, T7, T9, T14).
  5. Built but unwired — NowLine/nowTitle narration, ThinkingBubble, Ark Clipboard machine, anim-pulse/run-ring: shipped, tested, mounted nowhere (T18, T19, T21).
  6. Eval incentive design — the one opaque, ungated capability out-competes every typed tool, then renders worst (T17, feeds T10/T16).

Issues (T-numbers)

Wave 1 — replay & history identity

  • T1 Refresh during pending gate replays the whole turn. subscribeSession attaches to live runs at '-1' on top of a complete snapshot; client resets delta state and visibly re-streams. Gate itself survives (in-memory ask). Fix: atomic snapshot/offset boundary — capture each live run's durable offset inside the snapshot transaction and attach there (or attach tail-first and reconcile by offset). No content-dedup guards. Approval re-emit only if a repro shows the snapshot omits pending approvals. core/src/chat/subscribe.ts, run.ts, @tanstack/ai stream-durability/processor.
  • T11 Duplicate assistant paragraph. (A) same replay: re-delivered text deltas push a second text part after a tool group. (B) settledTranscript de-overlaps transcript vs stored by folded-text match on one user message — misfires duplicate whole spans with unmergeable positional h{n} ids. Fix: A rides on T1; B anchors the merge on stable native transcript record ids (never positional indices); until then, fallback constrained to contiguous boundary match. No content-hash dedup. core/src/chat/transcript.ts:102-127, harness/src/claude/history.ts:215-218.

Wave 2 — run lifecycle (one design)

  • T6 Stop slow and silent. No optimistic state (button bound to server-observed sessionGenerating); server settlement includes a sync readFileSync of the whole transcript; interrupt-and-resend serializes two stops. Fix: client stopping state + new SessionStatusKind member + timeout fallback; terminal RUN_FINISHED stays after authoritative settlement, but settlement shrinks (async transcript read, contextOccupancyFor off the critical path as a later usage update); optional immediate "stopping" ack chunk; redundant await rpc.chat.stop in send removed only with a concurrency test. core/src/chat/stop.ts, run.ts:313-396, client/src/chat-connection.ts:151-157.
  • TC FAILED badge + -- timer. FAILED currently means "≥1 errored/denied/nonzero-exit tool call in the last turn"; clock lives in in-memory Maps keyed by unstable message ids. Fix: persist run status/start/finish/terminal error on the run record; FAILED derives from RUN_ERROR only; chip reason from the terminal error (via title); timing keyed on persisted runId carried into snapshots. ui-kit-chat/src/store/session-status.ts, turn-rollup.ts, turn-clock.ts.

Wave 3 — approval policy (approval axis only)

  • T2 Over-prompting. Policy precedence deny > ask > allow + ask = "contains any shell metacharacter" makes the read-only allow-list unreachable for every compound command; classification is whole-string; cd missing; asks are one-shot with zero memory; decision RPC is a bare boolean. Fix: real parsing (sandbox parser/AST if exposed; else a deliberately narrow grammar — simple commands joined by &&/;/|, no quoted separators/substitution/redirects except 2>/dev/null, every segment head read-only, cd added); anything outside the grammar keeps asking. Session memory: permissionDecision gains scope: 'once' | 'session'; store records the exact normalized command string (never tool+first-token), applied only after hazardous-syntax rejection; card gains "Allow this exact command for the session". core/src/chat/gate.ts, ask.ts, contract, permission card.

Wave 4 — page-session correlation (held: coordinate with the in-flight fix from the other session; not on origin/main as of 2991888)

  • T20 Same eval renders twice (exec trace row + "Ran script on the page" card). parentToolCallId stamped only on the mcp.ts producer path; the chat run loop never stamps, so the grouper can't claim the parent exec. Fix: one canonical outer execution id stamped at the producer boundary on both paths; no heuristic correlation. Test matrix: chat+MCP paths, intervening text, multiple evals, missing metadata. core/src/chat/code-mode-parts.ts:30-33, core/src/chat/run.ts:250, core/src/api/mcp.ts:191, ui-kit-chat/src/store/page-session.ts.

Wave 5 — working-state UX (one decision covering both)

  • T18 RUNNING looks frozen + T19 silent long turns. Signals are on the wire (reasoning deltas, tool starts) and dropped: thinking-only chain segments render nothing (toolCalls > 0 gate); collapsed-trace line is a tally; the built narration components are mounted nowhere; spinner gates on submitted only. Fix, exactly two surfaces: (a) status-bar RUNNING chip gets a pulsing StatusDot (reduced-motion: static shape change, freeze-don't-vanish); (b) one narration line — extracted NowLine + nowTitle() fed from the existing Thread/grouping store (NOT an Activity.Root mount — that's a parallel store), pinned above the composer, gated on the working() union, sole live region. Thinking gate dropped with a non-empty-thinking condition; streaming compact line shows current activity. In-thread ThinkingSpinner retires. thread.tsx:96-202, now-line.tsx, now-title.ts, status-bar.tsx, chat-pane.tsx:109-111,272-274.

Wave 6 — trace rail rework

  • T4 active indicator wrong place / T12 stuck at header / T5 stray lines. No row identity; accent silently degrades from "running" to a scroll reading-band; segment maps to the inbound connector (off-by-one); collapsed trace draws an orphan corner; zero-height <li>s inject anchors. Fix (ordered, minimal): define segment ownership (row owns inbound, incl. last row); accent = running only, derived from tool status; reading-band spy loses the accent (quieter or removed); filter non-rendered rows from the anchor read; visibleRowAnchors.length === 0 draws nothing; resolved permission <li> unmounts. Update pinned tests trace-rail.browser.test.tsx:272,442,455. ui-kit-chat/src/styled/trace/rail.tsx.

Wave 7 — eval behavior + rendering (after wave 4; staged, measure between stages)

  • T17 agent prefers eval. Catalog search haystack omits keywords and the description (eval's "last resort" hint unreachable in the list the agent browses); page.snapshot summarized as accessibility-only; no reload capability; eval ungated; no page promptSnippet. Fix stage 1 (no new API): keywords + description into the haystack; snapshot summary rewritten around form/control state. Stage 2 (needs sign-off): approval: 'ask' on page.eval + concise promptSnippet. Stage 3 (deferred, needs sign-off): reload/navigation capability with defined completion semantics, no fake hard flag. core/src/chat/code-mode.ts:201, capabilities.ts, extensions/page/src/shared/defs.ts.
  • T10 raw escaped JSON. Exec card compact-stringifies result into an ansi block; JsonTree paths never reached; eval returns pre-stringified JSON. Fix: split logs (ansi) from typed result (structural renderer shared with ToolFallback's contentShape; objects/arrays → JsonTree; strings stay strings — no parse-twice heuristic); the JSON-string case fixed at the eval producer (coordinate with T17). core/src/cards/code-run-card.tsx:94-118.
  • T16 eval row truncated to "s". Fixed target 'script' starved by an uncapped value pill (max-w-40 loses to Chip's max-w-full; max-content track). Fix: eval target becomes the clipped first line of the script, value pill dropped; Chip maxWidth variant + minmax(0, max-content) only if another step type still reproduces. extensions/page/src/client/cards/session-steps.ts:16, session-card.tsx:85-115.

Wave 8 — independent UI/content fixes (parallelizable)

  • T3 "Denied by user" off-center. @pierre/diffs viewer reserves bottom scrollbar space in a symmetric frame. Fix: inspect rendered DOM for the real reserving element first (gutter-variable attribution unverified); likely bypass the diffs viewer for plain one-line xs output. Screenshot-verified; DOM-measurement tests banned. output-block.tsx, code-block.tsx.
  • T7 empty result frame. Frame decided from raw args, content by the embedded card. Fix: code-run emptiness predicate now; hasEmbeddedBody on the card entry as the structural contract. tool-call-card.tsx:58-97.
  • T8 "edits page" badge. Hardcoded MUTATING_BADGE. Fix: plain rename to the approved noun phrase — proposal "page edit" (copy needs sign-off); no per-def override surface. Update 4 pinning tests + stories. tool-presentation.ts:7.
  • T9 "1 action" misaligned. items-center on mixed mono/sans. Fix: baseline-align the title+metric text pair only (wrapper), controls stay centered; explicit line-heights. Apply in collapsible-card, tool-card METRIC, session-card HEAD.
  • T13 "Refresh the conversation" dead. Impl is attempt.abort(); identical snapshot; latched hydration; popover never closes. Fix: a refresh coordinator (owned where chat + queries meet) that reconnects and awaits refetch of session/markers/captures/usage; popover closes; inline progress; conversation stays (no skeleton re-arm). client/src/chat-connection.ts:158, panel.$sessionId.tsx:222-233.
  • T14 header padding asymmetric. h-15+items-center, no padding knob, inherited prose leading, raw literals. Fix: padding-driven layout + explicit leadings, existing --chat-text-* tokens, preserve total height; don't touch shared typography tokens.
  • T15 active tab invisible. Active = text color only; hover on inactive is strictly stronger. Fix: --chat-text-hi + --chat-fill + 2px accent inset box-shadow (layout-neutral); hover < active; test asserts aria-pressed + active modifier. status-bar.tsx:32-38.
  • T21 copy-button chaos. 8 hand-rolled implementations, 4 state machines, 4 failure policies; Ark Clipboard installed, unused; screenshot instance overlays content, text-chip, sr-only-only feedback. Fix: ui-kit-system gains a low-level clipboard state primitive (Ark machine) + separate TooltipIconButton+Swap presentation; Solid call sites migrate (inventory re-confirmed at dispatch); output-block actions move into chrome row, keyboard-visible; React site adopts the interaction contract; fumadocs' stays.

Sign-off gates before their waves run

  • T8 badge copy ("page edit")
  • T17 stage 2 (gate page.eval) and stage 3 (reload capability)
  • T20 coordination with the other session's in-flight fix

Full report with mechanisms, evidence and screenshots per issue: internal triage artifact (2026-08-23).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions