fix(conductor): one plan gate, required goal folder, rate-bounded dashboard trust - #6109
Conversation
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsThe single candidate describes the folder rate-limit bucket being keyed on the validated caller name ( I re-derived the rest of the diff independently: the No findings. [OPUS-REVIEWED] 4cdca99 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The unattended-patrol fix is partial: every cycle still blocks on the Watch
Suggestions
[DESIGN-REVIEWED] 4cdca99 |
GPT 5.6 Review — ✅ human override acceptedHuman judgment by @iamwhatever overrides the GPT 5.6 finding for This comment is updated in place on each push. The model was not re-run because an authorized human decision supersedes it. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of All claims verified: First-Principles-Verdict: PASS Three reported defects, each fixed at the declared level, every new guard derived from the untrusted-content boundary, and every alternative counted rather than assumed. What this change shipsIntent: make the conductor's first live-run failures not recur — plan without interrogating, file sessions under the goal, patrol without blocking on approvals. This is a FIX.
Watch
[FIRST-PRINCIPLES-REVIEWED] 4cdca99 |
Design Review dispositions
|
First Principles dispositions
|
cb43e4b to
dfd6b53
Compare
|
Two GPT-lane concerns, one per rationale. New head
|
…hboard trust Three defects reported from the first real conductor run. It interrogated the user before doing anything. Round 0 said "restate the plan, wait for the user", which left room for a clarification round ahead of the plan. Now the first reply IS the plan: everything the conductor can settle itself becomes a stated Assumption, at most one question is allowed and only when a wrong guess is unrecoverable, and a goal message that already authorizes execution dispatches round 1 in the same turn instead of re-asking for permission it holds. It created sessions with no folder. chat_folder_create sat inside the per-item loop, worded "once per goal (skip if it exists)" -- optional in tone and an extra prompted call, so it got dropped and the sessions landed loose at the sidebar's top level with nothing tying them to the goal. Hoisted to a round-level precondition ahead of the first session_create, with create-before-folder named as a defect and a fail-stop when the folder cannot be created or the session cannot be filed into it. Every session-control call prompted, which matters because patrol is autonudge-driven: it wakes itself and then blocks on an approval nobody is there to give. Dashboard verbs are now auto-approved BY NAME rather than by granting the server, under an invariant with two halves: a granted verb may CREATE or READ and never MUTATE pre-existing user-visible workspace state, AND its worst case in a loop must be bounded by the server. Per-call approval was the only thing rate-limiting a granted verb, and allowedTools has no argument or rate matching to replace it, so waiving the prompt removes the only brake on a loop. This adds that brake server-side, as a per-caller RATE on both creation verbs -- deliberately the guard that needs no durable state. A lifetime quota means nothing across a restart unless every rehydrate path carries its attribution, and there are 27 get_or_create_slot call sites; a five-minute window buys a restart one window rather than a clean slate. At the session rate, filling MAX_LIVE_SLOTS from empty takes over two hours of uninterrupted looping, every step visible in the sidebar. This cannot live in the conductor's own instructions: it reads untrusted text by design, and content that can drive it into a creation loop can equally override a "create at most N per round" line in its skill. Self-restraint is a convention; this is a control, enforced for every caller regardless of which agent is calling or what it was told. Folder creation is rate-limited for INTERNAL callers only. The endpoint is mixed-path -- the browser's own control posts to it, and a person organizing chats can legitimately create a dozen in one sitting -- and _audit_origin already tells the two apart. The bucket keys on the validated caller name rather than the session key: a dashboard: key naming a dead slot is refused, but a rotating non-dashboard key is not, so keying on it would let a caller earn a fresh budget per spelling. Behind the rate guard sit two capacity ceilings, both added here because neither resource had an adequate one. chat_folder_create had no bound at all, so a loop grew durable on-disk state without limit; MAX_CHAT_FOLDERS is tested inside mutate_folders, where len(folders) is authoritative, the same reason the parent is re-checked there. session_create had a global ceiling but no distribution, so one caller could hold all 500 slots and the person's own next chat tab got the 429; MAX_SLOTS_PER_CREATOR bounds one caller's share, with slots created through the verb attributed to their caller. A person's own tab and a fork are attributed to nobody and stay bounded by the global ceiling alone, so ordinary use never consumes an automated caller's share. Withheld and still mounted, so they pass the approval gate: chat_folder_move_session writes another session's folder_id -- the PATCH targets the session named in the ARGUMENTS while the strictly-resolved caller key is only the authority header, and mcp_dashboard calls it "the one tool here that writes to a session OTHER than the caller's"; session_send runs text as a peer's turn; session_stop discards a peer's in-flight work; chat_folder_move reparents an existing tree and no conductor step needs it. Per-verb refs are honoured on both backends, so the narrowing is real and not cosmetic: kiro-cli's is_tool_in_allowlist checks @server and then @server/<tool>, and allowed_tools_to_permissions maps the same entry to an exact KAS server/tool resource with no wildcard to widen it back. execute_bash stays ungranted for a different reason worth keeping distinct: allowedTools is name-scoped with no argument matching, so trusting the two bundled scripts cannot be told apart from trusting arbitrary shell.
dfd6b53 to
4cdca99
Compare
|
Both concerns fixed in
New tests: |
|
rebutted — the finding describes a chain that is fully present on
Every link of that chain is already auto-approved before this PR, through a grant this PR does not touch. On
So the read+send capability this finding is about predates the diff, and removing Two things I am not claiming. I am not claiming the residual is uninteresting: an agent that reads untrusted text, can read workspace transcripts, and can post to a shared channel is a real design property worth someone ruling on deliberately, so it is filed as #6154 rather than left implicit. And I am not claiming On the trust model, since it is the premise the whole grant list rests on: the conductor is an agent a person installs and invokes to run unattended, and its Span record: this is the 5th consecutive blocking finding in |
|
/ai-review override gpt 4cdca99: The read+send exfiltration chain is complete on origin/main without this diff — the conductor already holds web_fetch plus a whole-server @kirocrew-core grant, which registers get_chat_session (full workspace transcript read, broader than session_read_message) and send_message (outbound); removing session_read_message would not close the chain and would remove the only dashboard read the unattended patrol cycle needs. Pre-existing design property tracked in #6154. |
Human judgment recorded@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
Both concerns are correct, and the second one corrects a claim I made in this PR's own description. Taking them individually.
|
|
Verdict is PASS; answering both Watch items individually since each names something checkable.
|
Open PR relationship auditThis is a consolidated, point-in-time code-level audit note. It compares complete merge-base diffs and current/merged code; it does not treat a shared topic as duplication or partial coverage as completion. Relationship findings
No PR, Issue, label, branch, or review state was changed by the relationship-note portion of this audit. |
Problem / Motivation
Three defects observed on the first real run of the
kirocrew-conductoragentagainst a live goal:
Round 0 said "restate the goal, list the work items, wait for the user", which
left room for a clarification round ahead of the plan. The run spent its
opening turns asking questions rather than proposing a plan.
chat_folder_createwas step 1 insidethe per-item dispatch loop, worded "once per goal (skip if it exists)". The run
skipped it entirely and every dispatched session landed loose at the sidebar's
top level.
@kirocrew-dashboardwas withheld from the conductor agent's
allowedTools, sosession_create,session_send,chat_folder_*and the rest each prompted.Why it matters
Each one lands on the conductor's core loop rather than an edge:
before the plan inverts that, and every question about something the conductor
could have assumed is latency on work it could already have started.
With two or three items per round the user cannot tell which sidebar entries
are this goal's, and there is no grouping to close the goal out by.
monitor_start-driven: it wakes itself on a nudgeinterval. A cycle that blocks on an approval blocks on a person who is not at
the keyboard, so the loop stalls rather than patrols — the skill's own "Known
limits" section had to describe patrol as "attended-unattended" because of it.
What changed (motivation → approach → change)
1. Round 0 is now one plan message with one gate. The wording that permitted a
question round is replaced by an explicit rule: the first reply is the plan, and
anything the conductor can settle itself becomes a stated Assumption the user
can overrule in the same breath (which repo, items per round, which crew, how to
phrase an acceptance condition). A question is permitted only when a wrong guess is
unrecoverable and no default exists — credentials, spend, destroying someone's
work, or a goal too vague to name a single work item — capped at one and folded
into the plan message. A goal message that already authorizes execution ("just do
it", "go ahead", a re-send of a plan already shown) dispatches round 1 in that same
turn instead of re-asking for permission the conductor already holds.
2. The goal folder is a round-level precondition, not a per-item step.
session_createhas nofolderargument (SESSION_CREATE_SCHEMAatsrc/kiro_crew/validation.py:2628carries onlytitleandagent), so filing isa second call — which is exactly why an optional-sounding instruction inside the
loop got dropped. That second call is an earlier design choice, not a
constraint: adding a
folderfield would make filing atomic with creation anddelete this whole choreography. Doing that is wider than this fix, so it is
deferred to #6118 and the instruction layer is hardened in the meantime. It is hoisted above
the loop as a precondition ahead of the first
session_createfor the goal,session_create-before-folder is named as a defect rather than discouraged, and afailed
chat_folder_createstops the round instead of dispatching into no folder.The local GPT review lane then caught the sibling gap: creating the folder is not
sufficient, because the user can delete it mid-round, so a failed
chat_folder_move_sessionnow also stops the item before the seed — seedinganyway produces a session that is both unfiled and already doing work, strictly
worse than the outcome the precondition prevents.
3. Dashboard verbs are auto-approved BY NAME, under one invariant. The
dashboard server joins the same
_may_auto_approveceiling filter assession,reportand@kirocrew-core, so a governed host still withholds it (with theexisting
mcp_auto_approve_withheldSEL audit) and the derived KASpermissionsblock picks the change up automatically rather than restating it.
But the grant names verbs, not the server, because the first attempt at this
granted
@kirocrew-dashboardwhole and that was wrong.allowedToolsis the onepath that never reaches
hooks.on_tool_call, and the conductor ingests untrustedtext by design — its charter's worked example is "resolve this repo's open issues"
and it holds
web_fetchfor exactly that. So every auto-approved verb isreachable by content the conductor read, with no human in the loop on a
nudge-driven cycle. The rule that decides each verb, recorded on
_CONDUCTOR_DASHBOARD_GRANTSso the next reader extends it by rule:Half 2 is the one the review process forced, and it is not a restatement of half 1:
an unbounded create does damage without mutating anything. Per-call approval was
the only thing rate-limiting a granted verb, and
allowedToolshas no argument orrate matching to replace it, so waiving the prompt removes the only brake on a loop.
The brake has to live in the endpoint, and it comes in two layers.
The primary guard is a per-caller RATE on both creation verbs
(
dashboard/create_rate_limit.py): 20 sessions and 10 folders per caller per5-minute sliding window, refused with a 429. It is deliberately the guard that needs
no durable state. A lifetime quota means nothing across a restart unless every
path that rehydrates a session carries its attribution, and there are 27
get_or_create_slotcall sites across 12 modules; a five-minute window is notmeaningfully resettable — a restart buys a caller one window, not a clean slate. At
the session rate, filling
MAX_LIVE_SLOTSfrom empty takes over two hours ofuninterrupted looping, every step visible in the sidebar, which turns a silent
exhaustion into something a person watching the dashboard sees before it lands.
This cannot live in the conductor's own instructions. It reads untrusted text by
design, and content that can drive it into a creation loop can equally override a
"create at most N per round" line in its skill. Self-restraint is a convention; this
is a control, enforced for every caller regardless of which agent is calling.
Two details in that guard are load-bearing, and both are pinned by tests:
mixed-path: the browser's own "new folder" control posts to it, and a person
organizing their chats can legitimately create a dozen in one sitting, so
throttling them would be a regression with no security value.
_audit_originalready distinguishes the two — a request without the internal secret is the
browser.
key would give finer granularity but is partly caller-supplied:
_refuse_unattributable_callerrefuses adashboard:key naming a dead slot, butnot a rotating non-dashboard key, so keying on it would let a caller earn a fresh
budget per spelling. The caller name is checked against
_KNOWN_INTERNAL_CALLERS.The cost is that internal callers share one folder budget, acceptable when a goal
needs exactly one.
Behind the rate guard sit two capacity ceilings, both added here because neither
resource had an adequate one:
chat_folder_createhad no bound at all. Every other allocation path in thedashboard tests a ceiling; this one did not, so a loop could grow durable on-disk
state without limit. Now
MAX_CHAT_FOLDERS = 500, tested insidemutate_folders—len(folders)is only authoritative while the lock is held,which is the same reason the parent is re-checked and
orderrecounted there.session_createhad a global ceiling but no distribution.MAX_LIVE_SLOTS = 500bounds the total, so one caller could hold all 500 and every later create —the person opening a new chat tab included — got the 429. Now
MAX_SLOTS_PER_CREATOR = 50bounds what a single caller holds, with slots createdthrough the verb attributed to their caller (
_created_by); a person's own tab anda fork are attributed to nobody and stay bounded by the global ceiling alone, so
ordinary human use never consumes an automated caller's share. That attribution is
in-memory and resets on restart, which is precisely why the rate guard — not this
ceiling — is the control that has to hold adversarially.
Half 1 names that resource class on purpose rather than saying "any pre-existing
resource". A create always writes some shared bookkeeping — the slot table, the
folder index, the session-pulse counter — so a literal reading would forbid every
create and decide nothing. What it protects is state the person arranged and would
have to rebuild by hand.
chat_folder_treechat_folder_create_refuse_tree_shaping_if_unverifiablerefuses an unverifiable caller, and bounded by the newMAX_CHAT_FOLDERSsession_createMAX_LIVE_SLOTSand per caller by the newMAX_SLOTS_PER_CREATOR. One recorded side effect: it mints withorigin=SlotOrigin.USER, which is what keeps the session correctly private, and the session-pulse counter increments on that tag — so conductor sessions count toward the survey's "10 genuine user chats" window. A pre-existing conflation in the counter (it miscounts for every caller of the verb, including a hand-approved one), deferred as #6139session_read_messagechat_folder_move_sessionfolder_id— the PATCH targets the session named in the arguments; the strict caller key is only the authority header.mcp_dashboardcalls it "the one tool here that writes to a session OTHER than the caller's"chat_folder_movesession_sendsession_stopEvery withheld verb stays mounted (
@kirocrew-dashboardis still intools)— it just passes through
hooks.on_tool_calllike any ungranted tool. The cost isan approval when a round files a session, seeds a child, or stops one.
Being precise about when that cost lands, because the obvious framing overstates
the fix: on round 1 those prompts follow the human's plan approval, so someone is
present. On a multi-round goal they are not — round N+1 dispatches from the patrol
loop as items complete, so a filing or seeding prompt can land mid-patrol with
nobody there, which is the same stall shape defect 3 describes. So this PR reduces
the unattended-patrol stall rather than ending it: the read-only patrol cycle
(
chat_folder_tree,session_read_message) no longer prompts, and #6118 removes thefiling call, but a mid-patrol dispatch still blocks until then.
The same honesty applies to the evaluator.
execute_bashstays ungranted, andthe conductor's acceptance check runs
accept_eval.pythrough it, so every patrolcycle still pays one approval regardless of anything in this diff — the skill's own
cost note says so. Ending the per-cycle stall means giving the evaluator a non-shell
entry point (tracked in #6160), not widening this grant list.
The narrowing is real on both backends, not cosmetic — this was verified before
relying on it, since a per-tool ref that matched nothing would silently grant
nothing: kiro-cli's
is_tool_in_allowlistchecks@serverand then builds@{server}/{tool}and matches that (tool_permission_checker.rs), andallowed_tools_to_permissionsmaps the same entry to an exact KASserver/toolresource. No
kirocrew-dashboard/*wildcard is emitted, so nothing re-grants thewithheld verbs through the KAS path.
execute_bashstays ungranted for a different reason worth keeping distinct:allowedToolsis name-scoped with no argument matching, so trusting the twobundled scripts (
accept_eval.py,ledger_entry.py) cannot be distinguished fromtrusting arbitrary shell. There is no per-argument form of that grant the way there
is a per-tool form of the MCP one. The installer's docstring and the skill's cost
note both state what the code now does.
Tests
All in
test/test_conductor_agent.py, which already owned this installer'scontract — four of its existing tests failed against the permission change and were
updated rather than worked around:
The rate guard (
test/test_create_rate_limit.py, new — 8 tests):or one burst disables the verb permanently)
budget that same goal needs
up with a typo'd name is throttled to zero rather than silently unlimited
test_a_dispatch_round_for_a_real_goal_fits_in_one_window— the limits must notbind honest work: one folder plus twelve sessions in a window. Lowering the session
budget under ~12 fails this, which is the signal the limit has been tightened into
the legitimate path.
reason this does not reuse the never-evicting
AppRateLimiterThe two ceilings and the endpoint wiring (
test/test_chat_folder_cap.py, new, andadditions to
test/test_session_control.py):test_creation_is_refused_at_the_ceiling/test_the_last_folder_under_the_ceiling_is_still_allowed— the refusal and the boundary, so an off-by-one cannot cost the user their last
folder.
test_the_ceiling_is_tested_under_the_lock— a concurrent creator fills the treeinside the
mutate_folderscallback; hoisting the check above the lock makes thisland folder 501.
test_an_internal_caller_is_rate_limited_at_the_endpointandtest_the_browser_is_not_rate_limited— both directions of the mixed-path split.Applying the limiter unconditionally makes the second one 429.
test_a_created_slot_records_the_caller_that_asked_for_it— attribution, withoutwhich every caller's count stays 0 and the per-creator ceiling is decorative.
test_one_caller_cannot_consume_everybody_elses_slots— the hog is refused at itsown share while the global ceiling is nowhere near full, and a second caller still
succeeds.
test_slots_nobody_asked_for_are_charged_to_nobody— pins both directions: aperson's tab must not burn an automated caller's budget, and an empty creator key
must not match every unattributed slot.
The grant surface (
test/test_conductor_agent.py, 29 tests):test_only_create_and_read_verbs_are_auto_approved— pins the invariant byliteral verb name in both directions: the four withheld verbs absent, the four
granted present. Deliberately not asserted against
_CONDUCTOR_DASHBOARD_GRANTSitself, which would be tautological and could never catch a widening.
test_no_dashboard_verb_is_granted_outside_the_named_set— asserts the granteddashboard set is exactly those four and that the bare server never appears.
This is the guard the per-verb lists cannot give: a NEW tool added to the
dashboard server, or a stray tuple entry, fails here instead of being
auto-approved without anyone ruling on it.
test_no_write_tool_and_shell_not_preapproved— nofs_write, andexecute_bashmounted but never granted.test_grants_pass_through_the_governance_ceiling— a ceiling opinion on@kirocrew-corestrips that grant while leaving the ref mounted; the expectedlist carries the four per-verb dashboard grants.
test_kas_permissions_are_derived_from_the_filtered_grants— the dashboardgrants project to EXACT
server/toolresources, plus an explicit assertion thatno
kirocrew-dashboard/*wildcard appears. That wildcard is what would silentlyre-grant the withheld verbs on the KAS backend, so it is pinned rather than
assumed.
test_a_fully_governed_host_still_emits_the_permissions_key— the{"rules": []}branch, whose mere presence is what makes KAS load the spec, kept its coverage
when a ceiling on
@kirocrew-corealone stopped emptying the rule list.test_audit_failure_does_not_break_the_install— the spec still lands when theSEL sink raises.
test_skill_gates_the_plan_once_instead_of_interrogating,test_skill_requires_the_goal_folder_before_the_first_session,test_skill_states_the_real_approval_cost— doc ratchets in the same style asthe file's existing
test_skill_documents_artifacts_as_a_string_map, because fora skill it is the instruction, not the code, that regresses. The folder one also
pins the failed-move stop path; the cost one pins which verbs prompt, so the
skill and the spec cannot drift apart.
1986 tests pass across the changed surfaces (
test_session_control.py,test_chat_folder_cap.py,test_chat_folder_ownership.py,test_chat_folder_app_isolation.py,test_chat_folder_store_shape.py,test_chat_slot_create_folder.py,test_dashboard_chat.py,test_agent.py,test_kas_permissions.py,test_session_pulse_session_count.pyandtest_conductor_agent.py).Manual verification
N/A for the behavioural half — the fixes are instructions the model follows, and
exercising them means handing a live conductor a real goal, which is the reporter's
next step rather than something a unit test can stand in for.
Verified by hand for the permission half, in the direction that could have made the
narrowing a lie: per-tool refs are honoured on both backends, checked in source
before relying on them (
is_tool_in_allowlistin kiro-cli'stool_permission_checker.rs,_mcp_patterninacp/kas_permissions.py). There isno precedent in this repo for a per-tool
allowedToolsentry — every prior MCPgrant is whole-server — so a ref that matched nothing would have shipped "no grant
at all" while the description claimed a narrowed one.
Also verified the claim the withheld list turns on:
chat_folder_move_sessionPATCHes
/api/chat/slots/{target}/folderwhere the target comes from_resolve_chat_slot_key(args["session"], ...), not from the caller.The installed
~/.kiro/agents/kirocrew-conductor.jsonwas updated by hand to thesame result this installer produces, so the reporter can test on their current
install without waiting for a rebuild.
Local gates green on the pushed head:
test/test_conductor_agent.py, isort,flake8, mypy,
check_black_formatting.py,docs_lint.py,check_brand_name.py,check_harness_parity.py,check_changelog_history.py,check_focus_cue.py,check_loop_bound_locks.py. Local review ran both profile reviewers against theirextracted CI contracts on each revision.
Related Issues
no linked issue: reported directly from a live conductor run, not filed.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)