docs(hooks): document fail-closed PreToolUse exit contract (#7744) - #8402
Conversation
PR #7422 changed the PreToolUse script-hook exit-code contract: any exit that is neither 0 nor 2 now BLOCKS the tool (fail closed), where it previously only warned. Three places still described the old warn-only rule or under-pinned the new one: - docs/reference/kiro-cli/hooks.md: the Hook output bullets and the PreToolUse section now state 0=allow (stdout injected), 2=deny (stderr returned to the LLM), and any other exit (timeout, crash, unexecutable command) blocks the tool with detail preferring result.error, then stderr, then "exited with code N". Non-gating events keep warn-only semantics. A note points out there is no per-hook advisory/fail-open opt-out yet (#7547). - src/kiro_crew/hooks.py: the ScriptHook dataclass docstring now states the same contract instead of the pre-#7422 warn-only rule. - test/test_dashboard_approval.py: the two tests added by #7422 built their hook results with bare MagicMock, so a renamed production field would silently keep passing. They now construct real ScriptHookResult instances, so a field rename fails loudly as a TypeError. Documentation and test hardening only; no runtime behaviour changes. Closes #7744
Design Review (Fable 5) — ✅ PASSDesign-level review of Docs match the runtime ( Design-Verdict: PASS Docs and docstring now match the shipped fail-closed gate they had contradicted; test hardening is strictly stronger than the bare mocks it replaces. [DESIGN-REVIEWED] 5304ad1 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of All claims verified against the runtime. Every documented behavior matches the code at First-Principles-Verdict: PASS Docs said the opposite of shipped fail-closed behavior; every hunk here either corrects that or pins the contract the docs now describe. What this change shipsIntent: make the hooks reference and docstring stop describing the pre-#7422 warn-only rule that shipped behavior contradicts — a FIX.
WatchItem 5 rides along in a docs fix, but it is declared, targets the same contract the docs now describe, and leaves no siblings: grepped [FIRST-PRINCIPLES-REVIEWED] 5304ad1 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/hooks.py:3471 -- False positive or not applicable? A repository writer can comment: |
bolichen97
left a comment
There was a problem hiding this comment.
Docs now match shipped behavior, and I verified each claim against the runtime rather than the PR text: chat_runner.py's elif r.exit_code not in (0, 2) branch gates its block on event == HOOK_EVENT_PRE_TOOL_USE, and its detail is exactly r.error or r.stderr or f"exited with code {r.exit_code}" — the precedence the reference now documents. The subtler claim also holds: the exit_code == 2 branch above it has no event check, so stating that an exit 2 on a non-gating event surfaces as injected text rather than denying anything is accurate, and that was the easiest thing to get wrong here. The ScriptHook docstring correcting the stale pre-#7422 "Other: warning" rule closes the last place the old contract was still asserted. The test change is a genuine hardening rather than churn: building real ScriptHookResult instances makes a production field rename fail loudly as a TypeError, where the bare MagicMocks would have kept passing while configuring a stale attribute — and the note explaining why spec=/spec_set= cannot substitute (no-default dataclass fields are not class attributes) is correct.
Summary
PR #7422 changed the PreToolUse script-hook exit-code contract: any exit that is neither 0 nor 2 now BLOCKS the tool (fail closed), where it previously only warned. The user-facing hooks reference and the
ScriptHookdocstring still described the old warn-only rule — the opposite of shipped behaviour. This PR is documentation + test hardening only; no runtime behaviour changes.docs/reference/kiro-cli/hooks.md— the Hook output bullets and the PreToolUse section now state the shipped contract: exit 0 = allow (stdout injected), exit 2 = deny (stderr returned to the LLM), and any other exit (timeout, crash, unexecutable command) blocks the tool, with the block detail preferringresult.error, then stderr, thenexited with code N. Non-gating events (postToolUse, userPromptSubmit, stop) keep warn-only semantics. Added a plain note that there is no per-hook advisory/fail-open opt-out yet (tracked in PreToolUse hooks cannot deny on the subagent/task-runner paths (fire_tool_hooks is informational) #7547), and a clause that on non-gating events an exit 2 is not a gate (its marker is surfaced as injected text — matches the exit-2 branch inchat_runner.py, which has no event check).src/kiro_crew/hooks.py— theScriptHookdataclass docstring now states the same contract instead of the pre-fix(hooks): fail closed when a PreToolUse hook delivers no verdict #7422 "Other: warning" rule.test/test_dashboard_approval.py— the two tests added by fix(hooks): fail closed when a PreToolUse hook delivers no verdict #7422 built their hook results with bareMagicMock, so a renamed production field would silently keep passing. They now construct realScriptHookResultinstances, so a field rename fails loudly as aTypeError. The_blocking_hook_storehelper (same bare-mock pattern, same file) was hardened the same way. Note:MagicMock(spec=...)/spec_set=...was evaluated first and rejected — dataclass fields without defaults (hook_id,hook_name,event) are not class attributes, so mock's spec machinery rejects setting them; real dataclass instances are strictly stronger.Every doc claim was verified against the runtime implementation (
_fire()insrc/kiro_crew/dashboard/chat_runner.py, theelif r.exit_code not in (0, 2)fail-closed branch).Testing
isort --check-only src/kiro_crew test— passflake8 src/kiro_crew test— passmypy src/kiro_crew/— pass (no issues in 1279 files)scripts/check_black_formatting.py— pass (post-commit, diff-scoped)scripts/check_brand_name.py— pass (exit 0, no added-line hits)ScriptHookResultconstructions used by the hardened tests verified by direct instantiation (timeout/crash/exit-127/exit-0 kwarg sets), plus a negative check that an unknown field raisesTypeErrorpytestruns in CI (tests are not run locally on this host by policy)Pre-push review
Two model-pinned read-only review lanes on the staged diff: GPT lane PASS (no findings); Opus lane PASS (0 blocking, 2 Low advisories — both applied in this commit).
Closes #7744