fix(acp): stop error guidance naming commands that do not exist - #7929
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Claims verified: Design-Verdict: PASS Surface-blind formatter now emits surface-neutral prose, with the rule pinned in spec and tests — root cause addressed, proportionate scope. [DESIGN-REVIEWED] 022f563 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of All claims verified. First-Principles-Verdict: CONCERNS The two inert commands are gone and verified, but the author's own root cause — a surface-blind formatter naming one surface's affordance — has three counted siblings left in the same function. What this change shipsIntent: stop stuck users receiving repair instructions that do nothing on their surface — a FIX.
Watch
[FIRST-PRINCIPLES-REVIEWED] 022f563 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/acp/client.py:1808 -- " [GPT-REVIEWED] 022f563 False positive or not applicable? A repository writer can comment: |
06f9765 to
3c6761b
Compare
Disposition of First Principles round 1 (
|
3c6761b to
2858ca9
Compare
Disposition of First Principles round 2, plus a rebase (
|
Two branches of the surface-blind `_format_acp_error` told users to run commands that are not available to them. The malformed-request branch said to start "a fresh chat (`/chat new`)". `/chat new` exists nowhere in the product; the reset command is `/new` on Telegram and Discord, and a new tab on the dashboard. `/chat` in the frontend is a route path, not a slash command. The prompt-busy branch said to "send `!restart`", which is wrong three ways: it is a Slack-only bang alias, it is owner-gated even there, and it restarts the GATEWAY rather than the session. It was also unnecessary -- the handlers already reset and re-queue on AcpPromptBusy by themselves. This formatter has no channel context, and the malformed-request class is terminal, so a channel dispatcher hands the string straight to the user (see telegram `_user_safe_failure_reason`, which surfaces only `transient is False`). A stuck user therefore received exactly one actionable instruction and it was inert -- and because the session cannot build a valid request, typing the non-command forwards it as a prompt and re-fails with the same error. Both branches now name only a command every surface understands (`/compact`, which reaches the backend through the prompt transport even on Slack) and otherwise use prose. The advice survives; the fabricated commands do not. Both existing tests asserted `"/compact" in out or "/chat new" in out`. That disjunction let the fabricated command satisfy the assertion by itself, so neither would have reddened on a reintroduction; both are now tightened to `/compact` alone. New tests assert `/chat new` is absent at both handle raise sites and that `!restart` is absent from the busy branch, each paired with a positive check that the advice survived. Refs #7213 Refs #6022
2858ca9 to
022f563
Compare
Disposition of First Principles round 3 (
|
Disposition of First Principles round 4 (
|
iamwhatever
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (5 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: ACP error guidance printed remediation commands that do not exist, so the text is corrected to only name real commands; change is confined to acp/client.py plus its two error-surface tests. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/common/error-handling.md, docs/system-specs/modules/slack-gateway.md.
Problem / Motivation
Two branches of
_format_acp_errortold users to run commands that are notavailable to them.
Malformed request ended: start "a fresh chat (
/chat new)"./chat newexists nowhere in the product. The command to start a fresh conversation is
/newon Telegram and Discord, and a new tab on the dashboard;/chatin thefrontend is a router path (
<Navigate to="/chat">), not a slash command.Prompt busy said to "send
!restart". That is wrong three ways: it is aSlack-only bang alias (
slack/handler.py:159,slack/events.py:2555), it isowner-gated even there ("Only the owner can restart the gateway",
slack/events.py:745), and it restarts the GATEWAY rather than the session --it is registered as "restart the gateway (owner-only)" at
slack/events.py:829.So on both branches the actionable instruction a stuck user receives is inert.
Why it matters
The malformed-request class is TERMINAL (
_is_transient_raw_errorreturns Falsefor it), which is exactly why its text reaches the user verbatim: a channel
dispatcher surfaces a permanent ACP failure's own message instead of the generic
"please try again". Telegram's
_user_safe_failure_reasonis gated ontransient is Falseand does precisely this.That reader is, by construction, in a session that cannot build a valid request.
Typing
/chat newthere is not a no-op: it is not a command on any surface, soit is forwarded to the model as prompt text through the same broken session and
returns the identical
Improperly formed request. Following the guidancereproduces the failure.
/chat newappears in #7213's own "What did NOT fix it"list.
The
!restartadvice was also redundant: both the dashboard(
dashboard/chat_runner.py, retry-eligible branch) and Slack already reset andre-queue the session on
AcpPromptBusyby themselves. The user was being askedto intervene in a recovery that is automatic.
What changed (motivation -> approach -> change)
Symptom: repair guidance names commands that do not exist for the reader, on the
paths whose text is handed to users unmodified.
Root cause:
_format_acp_errorlives inacp/client.pyand cannot know whichsurface will render its output, but both branches were written as if the reader
were on a particular one.
Change: name a command only if every surface understands it, and otherwise use
prose.
/compactqualifies -- it reaches the backend through the prompttransport everywhere, including Slack, which additionally offers
!compactasits own alias. The malformed-request branch keeps
/compactand drops/chat newfor "start a new conversation". The prompt-busy branch names nothingand says the condition clears itself. The spec states the rule so the next edit
does not reintroduce a surface-specific spelling, and the comments point at it
rather than restating it.
Scope note, since #7213 stacks two defects: this PR fixes only the guidance,
not the malformation and not the retry policy. The retry half was already fixed
and merged by #7218, which classifies the string as terminal so the identical
payload is never re-sent -- that is why this text now reaches users at all. The
residue #7213 still carries is untouched here, which is why this PR uses
Refsrather than a closing keyword:
(silent fallback to the default vs. an explicit "pick a new model" prompt) --
a product decision, recorded as such by three triage passes on the issue;
Tests
Both pre-existing tests of the malformed branch asserted
"/compact" in out or "/chat new" in out. That disjunction is why the defectsurvived: the fabricated command satisfied the assertion on its own, so neither
test would have reddened on a reintroduction. Both are tightened to
/compact.test/test_acp_error_surface.py--test_malformed_request_guidance_names_no_unknown_commandandtest_prompt_busy_guidance_names_no_unknown_command, each over both handleraise sites (
_wait_for_responseand_dispatch_events). Each asserts thefabricated command is absent AND that the branch's advice survived, so deleting
a command cannot silently delete the guidance.
test/test_acp_true_provider_error.py--test_rewrites_into_actionable_prose, the same disjunction, tightened the sameway. It drives
_format_acp_errordirectly, so the two files cover theformatter and both raise sites.
Mutation-verified in both directions: restoring
a fresh chat (/chat new).reddens both parametrizations of the malformed test, and restoring
send \!restart`reddens both parametrizations of the busy test; each returns green on revert. 34 passed intest_acp_error_surface.py, 24 intest_acp_true_provider_error.py.flake8andisort` clean on every touchedPython file.
docs/system-specs/modules/slack-gateway.mdquoted the busy string verbatim andmoves with it, so no spec describes a message that no longer exists.
Manual verification
N/A -- unit coverage is the right bar. The change is two user-facing strings
built by a pure formatter, and the tests drive that formatter through both real
raise sites plus the direct entry point. Reproducing a live
Improperly formed requeston a Telegram-bound session is not available in thisrepo's test environment: pod security forces
telegram.enabled=False, so no liveTelegram round trip was attempted and none is claimed.
Related Issues
Refs #7213
Refs #6022
Pattern harvest
Rule candidate: review-prompt
Pattern: user-facing guidance names a command, flag, or path that does not exist
for the reader -- and a test disjunction (
assert A in out or B in out) lets thefabricated half satisfy the assertion by itself. Invisible to every automated
gate: these strings type-check, lint, and here satisfied two separate tests
through the
orbranch. Three generalizable checks: when a diff adds a commandname to user-facing prose, grep the product for that command AND confirm it is
available on every surface that can render the string (not just that it exists);
treat
orin an assertion about a single required affordance as a smell, sinceit makes the weaker half sufficient; and when a string is built somewhere with no
surface context, prefer prose over any one surface's spelling.