Skip to content

feat(conductor): ship the conductor agent + acceptance-evaluator skill as built-ins - #5652

Merged
iamwhatever merged 2 commits into
mainfrom
feat/conductor-builtin
Aug 25, 2026
Merged

feat(conductor): ship the conductor agent + acceptance-evaluator skill as built-ins#5652
iamwhatever merged 2 commits into
mainfrom
feat/conductor-builtin

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #5650 (session_send) — the skill's dispatch step sends seed prompts through that verb. Retargets to main automatically when #5650 merges.

Problem / Motivation

Kiro Crew can automate work inside one session, but a goal too large for one session — "triage N issues and publish fix PRs", "take this feature from design to merged PRs" — still requires the user to decompose it by hand, open a session per piece, paste context into each, poll them all, and decide every next step. The pieces to automate this already exist (#2435 session control, #5198 session ledger, #5650 session_send) but nothing assembles them into a usable role.

Why it matters

With dozens of concurrent sessions the marginal cost of supervision grows linearly: every open workstream is a tab the user must remember, re-read, and re-decide. The pain is no longer running parallel work — it is being the only coordinator of it.

What changed (motivation → approach → change)

Goal: one agent that owns a long-horizon goal and does only the four things a work item cannot do for itself — decompose, dispatch, verify, decide the next round. Approach: assemble the merged mechanisms rather than build new ones, and put the charter in the spec wherever a prompt sentence would otherwise be the only thing enforcing it.

Why not the dynamic-workflow engine. src/kiro_crew/workflows/ is the nearest existing mechanism — it already "orchestrates several Kiro Crew agents through ordered phases" — so it is worth saying why this is not a second spelling of it. A workflow is an authored script executed to completion: its phases are fixed before it starts, its agents are sub-agents rather than sessions the user can open, and there is no point at which a human re-plans it mid-run. The conductor is the opposite on all three: it dispatches top-level, steerable sessions the user can read and redirect while they run, it treats human_approval as a first-class acceptance kind, and re-planning between rounds is the normal path rather than a failure. A goal whose next round genuinely depends on what the last round produced cannot be a workflow script, because the script would have to be written before that information exists.

1. kirocrew-conductor agent_install_conductor_agent() in src/kiro_crew/agent.py, following the _install_research_agent pattern: generated at install time from build_agent_config(), so MCP invocations resolve to the machine's real paths (a static spec cannot ship). What the spec enforces structurally:

  • No fs_write. A task needing a write is a work item for a child session, never the conductor's own work.
  • No tool that can write a file. Not fs_write, and not code either — governance maps code to filesystem.write because it "writes files and can shell out" (platform/governance.py BUILTIN_TOOL_SCOPES), and it sits in WITHHELD_FROM_AUTO_APPROVE for the same reason. That is what makes "never does a work item's work itself" a property of the spec rather than of the prompt, and it is pinned by test_mounts_no_tool_that_can_write_a_file so the claim cannot outlive the tool list.
  • MCP surface narrowed to kirocrew-core + kirocrew-dashboard; inherited servers the charter has no use for are dropped. The builtin set is narrowed on the same rule — an unused grant is surface the charter cannot account for: web_fetch stays because the skill's worked example reads an issue list, while web_search, grep and glob are not mounted (fs_read covers every read the charter describes). tool_search is the one apparent exception and is load-bearing rather than unused: with MCP Tool Search active the session-control verbs are deferred, so without it the conductor cannot reach session_create / chat_folder_* / monitor_start at all. The prompt names that use explicitly.
  • Neither @kirocrew-dashboard nor execute_bash is in allowedTools. Auto-approve is the one path that never reaches the PreToolUse gate, so both stay off it and every session-control call — and every evaluator run — passes through hooks.on_tool_call where the deny floor and governance ceiling apply. Granting the opt-in dashboard set in this spec is the explicit per-agent assignment that set requires.
  • The grants that remain are filtered through the governance ceiling. allowedTools is built through _may_auto_approve() (may_skip_gate_now), the entry point whose whole purpose is that a new writer cannot reopen the bypass by restating a literal list. A ceiling with an opinion about @kirocrew-core now leaves it mounted but not pre-approved.
  • The KAS permissions block is derived from that filtered list, via allowed_tools_to_permissions(), instead of restating a blanket kirocrew-core/* allow. The rules come out identical on an ungoverned host, a later allowedTools edit carries through, and a ceiling that strips a grant strips its KAS rule with it. {"rules": []} when nothing qualifies, because the key's mere presence is what makes KAS load the spec.
  • The hand-built kirocrew-dashboard entry carries the managed-server metadata every entry build_agent_config emits gets: the conditional "type": "registry" marker and the KIROCREW_HOME env pin. Without the marker a registry-mode client silently drops the entry and the conductor's session-control tools never launch; without the pin the shim reads the default data home while the gateway runs under an override. Both helpers return empty on a default install, so the emitted spec is unchanged there.
  • CONDUCTOR_AGENT_FILENAME is registered in OWNED_KIRO_AGENT_FILES (src/kiro_crew/agent_files.py), the tuple whose docstring says every managed spec registers there. Absent from it, three consumers (the Playwright convergence sweep, doctor's dead-path repair, connection minting) classify Kiro Crew's own spec as a third-party file, so a dead resolved command path is reported as unfixable instead of repaired.

2. goal-conductor built-in skill (wheel-shipped under src/kiro_crew/builtin_skills/) carrying the operating procedure: the three work-item tests (independent / assertable / long-running), the conductor-vs-work-item boundary rule, dispatch steps (folder → session_createsession_send seed → ledger record), a patrol loop over monitor_start, two-phase acceptance for values that only exist after an item starts (PR numbers), ownership-signal respect during triage (claimed labels), and four stop conditions including an explicit needs-human exit.

  • Named goal-conductor, not conductor. That name is owned by the generated delegation skill (conductor_skill.generate_conductor_skill), and two existing paths delete <skills>/conductor/SKILL.md when agent.conductor_skill is false — the default: cli_setup.py on every setup run and the dashboard config handler on toggle-off. Sharing the name would let kirocrew setup erase the packaged skill on a stock install, and quarantine the user's delegation skill when the flag is on.
  • Active work-item state has a durable home. The procedure now records each open item as one compact item-<n> entry in the ledger's artifacts map — acceptance spec, session key, round, status, and read cursor — and rotates finished items out. The ledger is the only store that survives compaction and judging an item from its transcript is forbidden, so a spec held only in context is a spec patrol cannot rebuild. The encoding fits the field's real bounds (32 entries, key ≤128 chars, value ≤2000 chars), which are also what keeps it to pointers-plus-spec rather than prose.
  • The evaluator is invoked at the skill's resolved install path, not a hardcoded ~/.kiro/crew/skills/... — a KIROCREW_HOME override moves the skills root, so the hardcoded form would fail every call before patrol ran.
  • The per-cycle approval cost is documented rather than implied: each patrol cycle blocks on one approval for the evaluator, plus one per session-control call, so patrol is attended-unattended and the skill says to batch every open item into one evaluator call.

3. scripts/accept_eval.py (stdlib-only, Python 3.8+), the deterministic half of patrol: reads acceptance specs on stdin, emits pass|fail|pending|refused|error. Acceptance is never the model reading a child transcript — it is this script's verdict. Per-check timeout, evidence tail-capped, gh pr checks' pending exit code (8) mapped to pending rather than fail, decode pinned to UTF-8 with errors="replace" so a check's output cannot turn its verdict into a crash.

  • The security invariant is structural: no model-authored argv ever reaches subprocess. The evaluator accepts no command, argv array, or shell string from a spec. Every argv it runs is built here from a fixed template out of narrowly-typed fields, so pr_checks becomes gh pr checks <n> and nothing else executes. _SELF_BUILT_COMMANDS ({"gh"}) is the internal fail-closed assertion that this stayed true — it is not a spec-facing allowlist.
  • Why the allowlist was dropped rather than tightened again. An earlier revision of this PR carried a generic cmd kind behind an allowlist, and three review rounds established that no constraint on argv fixes the shape: bare interpreters on the list let python -c <payload> run anything; a basename-only check let /tmp/git execute an arbitrary binary under an allowlisted name; and then plain git reset --hard — allowlisted, and destructive. The third is decisive, because this script is invoked as an approved wrapper: Kiro Crew's denied-command floor reads the execute_bash string, which says python3 accept_eval.py, while the real argv arrives on stdin and runs with shell=False. A generic executor there removes the deny floor for whatever it accepts, so the bypass was the wrapper rather than the list.
  • Widening is additive and reviewable. Adding a capability means adding a kind whose handler builds its own argv (a pytest kind taking test paths, an npm_script kind taking a script name) — never a kind that accepts a command. The script's docstring states this as the supported path so the next contributor does not re-open the generic form. cmd is still recognised and answered with a refused verdict naming what to use instead, so a conductor carrying an older skill gets guidance rather than "unknown kind".
  • What the removal costs, and why it is acceptable. "The test suite passes" is no longer a local acceptance condition; it is expressed as pr_checks on the PR carrying the work, which covers it transitively because CI runs the suite. The skill's own worked example is already issue → PR → checks green.
  • A malformed item cannot hide its siblings. Item validation and id extraction moved inside the per-item try: a non-object entry ({"items": [1, {...}]}) raised on .get before the handler, aborting the run and discarding every sibling verdict. It now yields a positional error verdict.

Tests

test/test_conductor_agent.py — 22 tests. Installer: identity + charter; fs_write absent while execute_bash/@kirocrew-dashboard are reachable but not pre-approved; no tool the charter never names (web_search absent, web_fetch present); MCP narrowing; the dashboard entry carrying type/env under registry mode + home override and staying minimal without them; a ceiling-governed ref keeping its mount and losing its grant; permissions derived from the filtered list (collapsing to {"rules": []} when the grant is withheld); the withheld grant emitting one mcp_auto_approve_withheld SEL event, no event when nothing is withheld, and a raising audit sink still letting the spec land; CONDUCTOR_AGENT_FILENAME in OWNED_KIRO_AGENT_FILES; the packaged skill not colliding with the delegation skill's name; the ledger item encoding documented as a string map.

Evaluator — the invariant gets its own class: pr_checks builds its own argv (asserted exactly, with _run captured rather than executed); _run refuses a command it did not build; a source ratchet fails if any handler ever reads an argv/command/shell spec field, which is the regression a behavioural test cannot cover; pr_checks rejects a non-integer pr including bool. Behaviour: verdict vocabulary across the remaining kinds, the removed cmd kind refused with guidance naming pr_checks, a non-object item not aborting the run, malformed stdin as a clean exit 2.

Full run: 1400 passed / 3 skipped across the affected modules (agent, agent_files, skills, doctor dead-path, connections mint, session control, ledger, MCP dashboard, spawn roster). Gates green: black, isort, flake8, mypy, subprocess-encoding, brand-name, changelog-history, focus-cue, harness-parity, vendor-manifest, loop-bound-locks. accept_eval.py needs no security_posture registration (reads local state and prints; no redactor or egress calls).

Manual verification

After install, ~/.kiro/agents/kirocrew-conductor.json exists with resolved command paths. Open a session on the conductor agent and hand it a multi-item goal: it should present a plan before dispatching, seed children via session_send, evaluate acceptance through the bundled script (one approval prompt per evaluator call — by design), and surface non-assertable items instead of dispatching them. A logic dry-run against 10 real repo issues was performed and its two findings (two-phase acceptance, claimed-label exclusion) are folded into the skill. Note the runtime preconditions: agent.session_control must be true, and strict session identity must resolve for the reflexive tools.

Why no screenshot: backend-only — a generated agent spec, a wheel-shipped skill, a stdlib script and tests. No frontend path is touched and nothing renders differently.

Related Issues

no linked issue: this assembles already-merged mechanisms (#2435, #5198, #5650) into a new role; there is no filed issue it resolves. Those references are context, not closing trailers.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested a review from a team as a code owner August 24, 2026 17:31
@iamwhatever
iamwhatever requested review from Zedmor and removed request for a team August 24, 2026 17:31
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Aug 24, 2026
@iamwhatever
iamwhatever force-pushed the feat/conductor-builtin branch from 699b17a to 979b4f4 Compare August 24, 2026 17:39
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ human override accepted

Human judgment by @iamwhatever overrides the GPT 5.6 finding for d884a065487a59d7293ec7e07ebb89e6dfbfc679; the recorded reason is authoritative for this commit.

This comment is updated in place on each push.

The model was not re-run because an authorized human decision supersedes it.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt d884a065487a59d7293ec7e07ebb89e6dfbfc679: <one-sentence reason>

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of d884a065487a59d7293ec7e07ebb89e6dfbfc679 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Sound assembly of merged mechanisms, but per-cycle approval prompts undercut the unattended-coordination value the PR is built to deliver.

Watch

  • The stated harm is "the user is the only coordinator of dozens of sessions," yet patrol blocks on a human approval every cycle (evaluator run + each session-control call) — "a cycle that finds nobody at the keyboard waits rather than proceeding." The security reasoning for keeping execute_bash and @kirocrew-dashboard off allowedTools is correct and derived (the deny floor cannot see inside an approved wrapper), but it means the role trades reading N sessions for clicking N prompts; whether that trade is a win in practice should be validated before the role is promoted further, and a purpose-built pre-approvable acceptance path (e.g. an MCP verb rather than a shell wrapper) is the eventual escape hatch.
  • All conductor runtime state lives as hand-serialized JSON strings inside the ledger's general-purpose artifacts map, with the encoding, rotation, and 32-entry/2000-char discipline enforced only by SKILL.md prose. One malformed write by the model silently loses an item's acceptance spec (the skill's own remedy is "re-derive it from the child session"). Acceptable to ship, but if this role sticks, item state deserves a typed store rather than a prompt-maintained protocol.

[DESIGN-REVIEWED] d884a06

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Aug 24, 2026
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed d884a065487a59d7293ec7e07ebb89e6dfbfc679 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] d884a06

Verdict parsed from the review's SHA-scoped output markers for commit d884a065487a59d7293ec7e07ebb89e6dfbfc679.

False positive or not applicable? A repository writer can comment:
/ai-review override fable d884a065487a59d7293ec7e07ebb89e6dfbfc679: <one-sentence reason>

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of d884a065487a59d7293ec7e07ebb89e6dfbfc679 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All claims verified against the repo. Summary of the counts I ran:

  • kind == "cmd" / "kind": "cmd" across the tree: only accept_eval.py itself and its own test — the kind never existed in any shipped version; it was removed during this PR's own review rounds.
  • Existing acceptance-evaluation mechanism: none (accept_eval|acceptance grep finds unrelated hits; workflow gates are CI invariants, not runtime acceptance).
  • The MCP-first rule at docs/architecture/mcp.md:475 and AGENTS.md name exactly one allowed bash-harness exception (kirocrew computer call).
  • Name-collision claim confirmed (cli_setup.py:329-334 deletes <skills>/conductor/); ledger bounds confirmed (session_ledger.py:75-77, validation.py:1076); artifacts_not_string_map confirmed; withheld-grant SEL event has 10 sibling emitters; UNADVERTISED_AGENTS already reserved kirocrew-conductor at base.

First-Principles-Verdict: CONCERNS

The evaluator ships as a shell script the conductor must get approved every patrol cycle, when the repo's own MCP-first rule provides the promptless, path-stable spelling.

What this change ships

Intent: let one agent own a goal too big for one session — decompose, dispatch sessions, verify, re-plan. ADDITION.

  1. New kirocrew-conductor agent installed on every config rebuild — justified
  2. New goal-conductor skill shipped to every install — justified
  3. Acceptance evaluator runs as a bash script, one approval prompt per patrol cycle — duplicate of the MCP-tool path (mcp_core.py)
  4. Session-control and shell calls always prompt (nothing auto-approved) — justified, derived
  5. Withheld auto-approve grants leave an SEL audit record — justified (10 sibling emitters)
  6. Conductor spec self-repairs via OWNED_KIRO_AGENT_FILES — justified, documented convention
  7. Evaluator answers a cmd spec with a guidance refusal — zero consumers
  8. Ledger artifacts becomes the durable work-item store — justified, fits existing bounds

Watch

  • The bash-script evaluator contradicts a documented invariant and its own pitch. docs/architecture/mcp.md:475 mandates MCP-first because "kiro-cli may refuse to run a CLI command via bash," and AGENTS.md allows exactly one bash exception; this adds a second, undeclared as such. The cost is real and admitted: "each patrol cycle blocks on one approval… a cycle that finds nobody at the keyboard waits" — while the description sells relief from being "the only coordinator." Since no spec field reaches the argv (the fixed gh template is the boundary), the per-call prompt bounds nothing the script doesn't already bound. An accept_eval tool on kirocrew-core — already mounted and auto-approved in this spec — removes the prompt, the <this skill's dir> path-resolution fragility paragraph, and execute_bash from the spec entirely.
  • The cmd tombstone has zero consumers. "a conductor carrying an older skill gets a message" — no older skill exists; the kind died inside this PR's review rounds (grep kind == "cmd": the script and its own test only). The unknown-kind error verdict already refuses safely.

Subtractions

  • Replace accept_eval.py + shell invocation with an MCP tool on kirocrew-core (mcp_core.py), then drop execute_bash from the conductor's tools — the spec ends up with no shell at all, a strictly smaller surface than "shell, but every call prompts".
  • Delete the kind == "cmd" branch in accept_eval.py and test_the_cmd_kind_is_refused_and_says_what_to_use — 0 shipped consumers of the kind; the unknown-kind error already refuses.

[FIRST-PRINCIPLES-REVIEWED] d884a06

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Aug 25, 2026
@iamwhatever
iamwhatever force-pushed the feat/conductor-builtin branch from 979b4f4 to 79ee449 Compare August 25, 2026 07:17
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Aug 25, 2026
@iamwhatever

Copy link
Copy Markdown
Collaborator Author

Head is now 79ee449c4a7136986ec092c9a99f9b1a3b186b5c (rebased onto the current feat/session-send tip, which was force-pushed under this branch). One disposition per finding.

  • src/kiro_crew/agent.py:4402 — static auto-approvals bypass governance — fixed

The finding holds as stated, and it holds against this repo's own stated invariant, not just in the abstract: allowedTools is the one path that never reaches hooks.on_tool_call, and may_skip_gate_now's docstring exists because five prior writers each independently produced exactly this state. A static ["session", "report", "@kirocrew-core"] in a new installer was a sixth. The grant list is now built through _may_auto_approve() (the module-local delegate to may_skip_gate_now), so a ceiling with an opinion about a ref leaves it MOUNTED in tools and off allowedTools, and the gate applies the real per-tool rule with real arguments. I also took the blanket MCP permission out by derivation rather than deletion: permissions now comes from allowed_tools_to_permissions(config["allowedTools"], ...) instead of a hand-written kirocrew-core/* allow, so the same ceiling that strips the grant strips its KAS rule with it — a hand-written block would have survived the filter on the KAS backend. Falls back to {"rules": []}, since the key's presence is what makes KAS load the spec at all. Pinned by test_grants_pass_through_the_governance_ceiling and test_kas_permissions_are_derived_from_the_filtered_grants.

  • src/kiro_crew/builtin_skills/.../accept_eval.py:82 — model-authored specifications execute unvalidated commands — fixed on the reachable substance, rebutted on the prescribed remedy

The reachable half is real and is fixed: with python/python3/node/npx on the allowlist, a model-authored spec could name python -c <payload> and the list was decorative. Those four are gone; ALLOWED_COMMANDS is now pytest, gh, git, npm, make, cargo, go. I did not take the prescribed fix — "remove generic cmd and file evaluation" — for two separate reasons. cmd IS the verdict carrier the evaluator exists for (a work item's acceptance is "this test suite exits 0"); removing it removes the feature rather than the hole, which is disproportionate to a spec whose every invocation already prompts. And file executes nothing at all — it is Path(path).exists(), a read-only existence check with no subprocess on that path — so it carries none of the consequence the finding describes. What I did instead of leaving the gap implicit is stop overclaiming: the docstring previously said "the allowlist — not the model's judgment — is what bounds what can execute", which is false while make, npm, cargo and go run project-authored code (a Makefile recipe, an npm script, a build.rs). The script, the skill and the PR description now say the allowlist NARROWS the surface and is not a sandbox, and name the control that actually bounds execution: execute_bash is deliberately kept off allowedTools, so every evaluator run passes the tool-call hook and prompts. test_bare_interpreters_are_refused pins the exclusion.

  • src/kiro_crew/builtin_skills/.../accept_eval.py:144 — malformed item crashes the evaluator — fixed

Confirmed exactly as described: item_id = str(item.get("id", "?")) sat outside the per-item try, so {"items": [1, {"id": "valid"}]} raised AttributeError before the handler and took every sibling verdict down with it — which is the one failure mode this script's contract says it must not have. Validation and id extraction moved inside the guard, with an explicit non-dict check and a positional #<n> fallback id so a malformed entry stays identifiable in the output instead of being silently unlabelled. test_a_non_object_item_does_not_abort_the_run asserts the malformed entries get error verdicts while the valid sibling still reports pass.

@iamwhatever

Copy link
Copy Markdown
Collaborator Author

Head is now 79ee449c4a7136986ec092c9a99f9b1a3b186b5c. One disposition per finding.

  • BLOCKER: skill-path collision with kirocrew.conductor_skillfixed, by vacating the name

Verified end to end before acting, because the consequence is install-breaking rather than theoretical: generate_conductor_skill writes skills_loader._dir / "conductor" / "SKILL.md", _ensure_builtin_skills syncs builtin_skills/<name>/ to base / <name>, and agent.conductor_skill defaults to False at config/loader.py:6936. So on a stock install cli_setup.py's skill_path.unlink() deletes the packaged SKILL.md the agent's own prompt tells it to read, leaving an orphaned scripts/ dir and a provenance marker; with the flag ON, the delegation skill's content diverges from the packaged tree and the builtin sync quarantines the user's skill to conductor.user-backup. Both halves are real. Of the two subtractions you named I took the second — vacate the name: the packaged skill is now goal-conductor, so the two mechanisms no longer contend for one path and neither existing deletion path can reach it. I did not fold the procedure into _CONDUCTOR_SYSTEM_PROMPT because that variant does not actually work: _iter_skill_files discovers a skill BY its SKILL.md, so a scripts/-only directory is never synced and accept_eval.py would never reach the user's skills dir — the script delivery rides on the very file that variant deletes. The rename keeps the delivery mechanism and still leaves the conductor name to its existing owner. test_builtin_skill_does_not_collide_with_the_delegation_skill pins the non-collision, and the installer docstring records why the name is what it is so the next person does not re-take it.

  • WATCH: the allowlist claim is contradicted by its own entries — fixed (claim corrected, entries narrowed)

Agreed, and the zero-option framing is the sharpest way to put it: with python/python3/node/npx present, "no allowlist" was observably equivalent for a hostile spec. Both halves are addressed rather than only the wording. The bare interpreters and npx are gone from ALLOWED_COMMANDS. The claim is restated where it appeared — the script docstring, the skill's "Known limits", and the PR description — to say the list narrows the surface and is not a sandbox, and to name what does bound execution (execute_bash off allowedTools, so every run prompts). I deliberately kept make, npm, cargo, go: they also run project-authored code, so shrinking to them alone would not have made the original claim true, and removing them would delete the verdict carriers the evaluator exists to run. Honest documentation of a real bound beats a smaller list still described as a boundary it is not.

  • WATCH: CONDUCTOR_AGENT_FILENAME not in OWNED_KIRO_AGENT_FILESfixed

Correct, and the docstring of that tuple already states the contract the new spec was violating ("adding a new managed agent spec is a one-line change in ONE place"). Added. Your consequence chain matched what I traced in the three consumers: doctor_deadpath classifies the file managed=False, records it as foreign_dead, skips the repair/re-verify path, and kirocrew doctor exits nonzero calling Kiro Crew's own spec an unfixable third-party file. Kept out of REQUIRED_KIRO_AGENT_FILES on purpose — the installer degrades to logger.debug and only its own feature is lost — and I updated that block's parenthetical so the exclusion reads as a decision rather than an oversight. test_spec_is_registered_as_kirocrew_owned pins membership.

  • SUBTRACTION: replace the permissions literal with allowed_tools_to_permissions(...)fixed

Taken, and it turned out to compose with the GPT lane's governance finding rather than merely tidying: deriving from the FILTERED allowedTools means the ceiling reaches the KAS block too, where the hand-written kirocrew-core/* allow would have survived it. Same rules on an ungoverned host, tracks future allowedTools edits, {"rules": []} when nothing qualifies so the spec stays loadable by KAS.

  • SUBTRACTION: shrink ALLOWED_COMMANDSfixed (same change as the allowlist watch above; recorded separately so this subtraction is not left looking unanswered)

The four entries you named (python, python3, node, npx) are removed. I did not shrink further, for the reason given in that item: the remaining entries are verdict carriers, and the residual risk they carry is now documented rather than denied.

  • Inventory item 4 (dashboard calls always prompt) — additional gap fixed beyond what was flagged

Your inventory judged this one justified, and it was — but auditing it surfaced that the same reasoning had not been applied to execute_bash, whose per-cycle approval was undocumented. The skill now states the steady-state cost concretely (one approval per patrol cycle for the evaluator, plus one per session-control call) and says to batch every open item into a single evaluator call. Recording it here because the honest version of item 4 is what made the omission visible.

@iamwhatever

Copy link
Copy Markdown
Collaborator Author

Advisory CONCERNS, answered rather than left to a green check. Head is now 79ee449c4a7136986ec092c9a99f9b1a3b186b5c.

  • WATCH: "the safety story overstates its mechanism" — fixed

This is the finding I most agree with, and it was the one a green rollup would never have forced. The PR claimed the charter is "enforced structurally, not just by prompt: no fs_write" while the spec grants execute_bash and the evaluator's allowlist carried python, node, npx, make, cargo, go — so the sentence advertised a boundary that did not exist, and a pinned test called it a "security property". Both halves of your either/or are now done, deliberately: the allowlist is narrowed (bare interpreters and npx removed) AND the claim is restated, because narrowing alone would not have made the original sentence true while make/npm/cargo/go still run project-authored code. The script, the skill and the PR body now say the list narrows the surface rather than bounding a hostile spec, and name the real control — execute_bash withheld from allowedTools, so every evaluator run prompts. The SKILL.md self-contradiction you were reading around ("This spec has no fs_write and no execute_bash" three paragraphs above "execute_bash is granted") is also gone.

  • WATCH: unattended patrol vs. per-cycle prompts — fixed as documentation, not as behaviour

The mechanism you describe is exactly right and I am not changing it: execute_bash stays off allowedTools for the same reason @kirocrew-dashboard does, so the prompt is the control, not an oversight. What was wrong was that the skill documented this only for dashboard calls and left the bash prompt — the steady-state cost of every single patrol cycle — unstated. "Known limits" now names it concretely: each cycle blocks on one approval for the evaluator plus one per session-control call, patrol is therefore attended-unattended (the loop wakes itself, but a cycle with nobody at the keyboard waits rather than proceeding), size the nudge interval for that, and batch every open item into ONE evaluator call rather than one call per item. The problem statement's "removes per-step supervision" claim is narrowed accordingly in the PR body.

  • SUGGESTION: expose the evaluator as a kirocrew-core MCP tool — needs-a-decision

I think you are right on the merits and I am not taking it in this PR, so I am putting the question rather than filing it. It genuinely dissolves both concerns at once: no execute_bash grant, so the no-write charter becomes structural instead of prompt-deep, and a deterministic read-only check is safely auto-approvable, which removes the per-cycle prompt that makes patrol attended. It is also the repo's stated MCP-first rule for LLM-facing capabilities, so the current shape is the deviation. What stops me doing it here is that it is a different shape of change, not a bigger version of this one — a new kirocrew-core tool with its own schema, governance surface, registration ratchet and tests, landing on a PR that is stacked under #5650 and already carries the agent spec plus the skill. Filing it as an issue would be the wrong disposition: the choice between "ship the bash-invoked script now and migrate" and "hold the conductor until the tool exists" is a maintainer call about what ships, and an issue asking that question sits unactionable. So: which do you want? I default to landing this as-is and migrating the evaluator to a core tool in a follow-up, on the grounds that the current form is honestly documented and the prompt is a real control rather than a missing one.

@iamwhatever

Copy link
Copy Markdown
Collaborator Author

Advisory FINDING on a passing check, answered. Head is now 79ee449c4a7136986ec092c9a99f9b1a3b186b5c.

  • src/kiro_crew/agent_files.py:36kirocrew-conductor.json absent from OWNED_KIRO_AGENT_FILESfixed

The consequence chain checks out step for step against doctor_deadpath.check_dead_paths: not in the tuple → managed=False → recorded as foreign_dead → never enters the repair/re-verify path → kirocrew doctor exits nonzero reporting Kiro Crew's own generated spec as an unfixable third-party file. And the trigger is not exotic for this spec in particular, since its MCP commands are sys.executable-derived absolute paths written at install time, which is exactly what a Python upgrade or a venv move invalidates. Applied the fix as prescribed — CONDUCTOR_AGENT_FILENAME added to the tuple — and left it out of REQUIRED_KIRO_AGENT_FILES on purpose, since the installer degrades to logger.debug and only its own feature is lost; I updated that block's parenthetical so the exclusion reads as a decision rather than an omission. test_spec_is_registered_as_kirocrew_owned pins membership so the next managed spec cannot repeat it.

@iamwhatever
iamwhatever force-pushed the feat/conductor-builtin branch from 79ee449 to 22e2b38 Compare August 25, 2026 07:23
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Aug 25, 2026
@iamwhatever
iamwhatever force-pushed the feat/conductor-builtin branch from 22e2b38 to fcd0cde Compare August 25, 2026 16:29
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Aug 25, 2026
@iamwhatever
iamwhatever force-pushed the feat/conductor-builtin branch from 5276ebb to e637b0d Compare August 25, 2026 19:27
@iamwhatever

Copy link
Copy Markdown
Collaborator Author

Head is now e637b0dad1333e69cab37385d83d476bd0b63a88. Both blocking findings are rebutted on maintainer judgment and the migration that removes their cause is tracked in #5926; the two advisories are fixed. A formal /ai-review override follows for the current head.

  • accept_eval.py:113 — bare gh resolution executes agent-planted binaries — rebutted

The reasoning is sound but the change it demands is not the right trade, and your own labelling agrees: this is anchored residual/security, not backend-security-controls. Reaching it requires an actor who can already write a gh earlier on PATH than the real one — and anyone who can do that owns the user's interactive shell too, so the evaluator is not the boundary being crossed. The control that actually stands between a spec and execution is the per-call approval on execute_bash, which the conductor's spec keeps deliberately un-auto-approved. The prescribed fix ("require a validated absolute gh") also needs a trusted-path resolution mechanism that a stdlib-only script outside the package cannot own, so taking it would trade this residual for a hand-rolled one. Left as-is, recorded rather than hidden.

  • accept_eval.py:160file acceptance bypasses sensitive-path enforcement — rebutted (maintainer override; residual accepted, not disputed)

Stating this precisely, because it is the one place my own reading differs from the ruling and the record should show that rather than launder it. The mechanism holds: hooks.py:554 runs is_sensitive_path on every target for every tool, it is documented as "always enforced, before all other checks", its denial is phrased as access rather than read, and the repo already added is_sensitive_bash_command specifically to close the shell spelling of the same probe — so the evaluator's stdin-JSON spelling is a third route past a floor this repo has twice chosen to close. What the route yields is a boolean: whether a path exists, never its contents. The maintainer has weighed that against the cost of a fourth in-place patch on this file and accepted it as a residual for the life of this PR, on the grounds that the evaluator is human-approved on every call and that #5926 removes the whole class by moving the evaluator inside the package, where is_sensitive_path is directly callable. I would have narrowed the file kind to project-relative paths instead — roughly ten lines, no security list duplicated — and I am noting that as the cheaper alternative if a reviewer wants it before merge rather than after. Not disputed as a false positive; accepted as a known residual with a filed exit.

  • SKILL.md:89"inherit the default otherwise" inherits kirocrew-conductorfixed

A real functional bug, not a wording nit, and worth spelling out because the failure is silent: leaving agent unset inherits the conductor's own spec, which deliberately has no fs_write, so a child dispatched to write a file cannot — and the item reads as stalled rather than misconfigured. The step now says to call select_crew and pass the agent it names (the matched crew, or the default_agent it returns), with an explicit "do NOT leave agent unset to inherit the default" and the reason, so the next reader cannot re-derive the old shortcut.

  • SKILL.md:145 — two-phase acceptance contradicts the evaluator's error verdict — fixed

Correct, and an internal contradiction rather than an external one: the skill promised pending for a TBD item while pr_checks returns error for a non-integer pr. Fixed in the direction you prescribed — an item whose value is not yet known is left out of the evaluator batch entirely and tracked as waiting in the conductor's own bookkeeping, rather than sent with a placeholder. I also recorded why the evaluator does not treat a missing field as pending: doing so would make a genuinely malformed spec indistinguishable from one that is merely early, which is the failure the strict verdict exists to prevent.

@iamwhatever

Copy link
Copy Markdown
Collaborator Author

/ai-review override gpt e637b0d: maintainer accepts both as residuals — the planted-gh path needs pre-existing PATH write access and every evaluator call is human-approved, and the file-existence probe yields a boolean whose whole class is removed by moving the evaluator into kirocrew-core (tracked in #5926).

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Aug 25, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for e637b0dad1333e69cab37385d83d476bd0b63a88.

maintainer accepts both as residuals — the planted-gh path needs pre-existing PATH write access and every evaluator call is human-approved, and the file-existence probe yields a boolean whose whole class is removed by moving the evaluator into kirocrew-core (tracked in #5926).

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Aug 25, 2026
@iamwhatever
iamwhatever force-pushed the feat/conductor-builtin branch from e637b0d to d884a06 Compare August 25, 2026 20:12
@iamwhatever

Copy link
Copy Markdown
Collaborator Author

/ai-review override gpt d884a06: maintainer accepts both accept_eval.py findings as residuals — the planted-gh path needs pre-existing PATH write access and every evaluator call is human-approved, and the file-existence probe yields a boolean whose whole class is removed by moving the evaluator into kirocrew-core (tracked in #5926).

@iamwhatever

Copy link
Copy Markdown
Collaborator Author

Head is now d884a065487a59d7293ec7e07ebb89e6dfbfc679.

  • BLOCKER: code is mounted on an agent whose stated structural property is that it cannot write — fixed

Correct, and it is the sharpest kind of finding because it caught the author's own rule being applied inconsistently. Verified rather than taken on trust: platform/governance.py's BUILTIN_TOOL_SCOPES comment says code "writes files AND can shell out" and maps it to ("commands", "tools", "filesystem.write"), and acp/kas_permissions.py lists it in WITHHELD_FROM_AUTO_APPROVE for the same reason. So while code was mounted, the sentence the whole charter rests on — "nothing that lets it do a work item's work itself" — was simply false, and the docstring asserted it anyway. You also identified the right precedent: I applied exactly this "an unused grant is surface the charter cannot account for" test one round earlier to drop web_search, and pinned it in test_mounts_no_tool_the_charter_never_names, then missed the larger grant sitting three lines above it. "code" is deleted from _install_conductor_agent()'s tools, the installer docstring now says no tool that can write a file and names code explicitly as the trap rather than only fs_write, and the agent's own prompt states the no-write property as a property of its spec instead of a rule it is asked to follow. test_mounts_no_tool_that_can_write_a_file pins fs_write and code together so the next reader cannot re-add one and leave the claim standing.

  • WATCH / SUBTRACTION: drop grep and globfixed

Agreed, same test, no argument: 0 references in _CONDUCTOR_SYSTEM_PROMPT or goal-conductor/SKILL.md, and fs_read covers every read the charter actually describes. Both dropped, and the negative is now asserted alongside web_search so the three travel together.

  • WATCH / SUBTRACTION: drop tool_searchrebutted, and taking your "or name a use" branch instead

This one is load-bearing rather than unused, which is why I took the second half of your own prescription. With MCP Tool Search active the session-control specs are deferred: session_create, session_send, session_read_message, session_stop, chat_folder_*, session_ledger_* and monitor_start are not in the agent's tool list until it loads them by id, and a first direct call fails with "a tool with the name … does not exist". Dropping tool_search would therefore not trim surface — it would remove the conductor's only route to its entire dispatch and patrol path, which is the feature. The defect you found was real but it was in the prose: nothing named the use. The prompt now spells out that those verbs are deferred, what the failure text means, and that this is what tool_search is mounted for; test_mounts_no_tool_the_charter_never_names asserts both that it stays mounted and that the prompt names it, so it can no longer sit there unexplained.

  • SUBTRACTION: collapse the cmd tombstone in accept_eval.py:_evaluaterebutted on proportionality; your reasoning is accepted, the change is not taken here

Your factual claim is right and mine was wrong: cmd never shipped — it existed only in this PR's own review history — so the "a conductor carrying an older skill" consumer count really is 0, and I should not have justified the branch that way. What stops me taking the subtraction is where the code lives. accept_eval.py is the span that took four consecutive blocking rounds this PR (interpreters on the allowlist → argv[0] path bypass → approved-wrapper bypass of the denied-command floor → planted-gh plus the file-existence probe), and the maintainer closed it by formal /ai-review override rather than a fifth patch. Reopening that file to delete six lines of dead code re-arms every lane on the exact surface that was just settled, for zero behavioural change. The tombstone is inert: it returns a refused verdict for a kind nothing emits. It goes with the evaluator when #5926 moves it into kirocrew-core, where the whole _evaluate dispatch is rewritten anyway. Recorded as a deliberate deferral of a correct observation, not as a disagreement with it.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for d884a065487a59d7293ec7e07ebb89e6dfbfc679.

maintainer accepts both accept_eval.py findings as residuals — the planted-gh path needs pre-existing PATH write access and every evaluator call is human-approved, and the file-existence probe yields a boolean whose whole class is removed by moving the evaluator into kirocrew-core (tracked in #5926).

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Aug 25, 2026
Base automatically changed from feat/session-send to main August 25, 2026 20:47
@iamwhatever
iamwhatever enabled auto-merge (squash) August 25, 2026 20:49
@iamwhatever

Copy link
Copy Markdown
Collaborator Author

Verdict moved BLOCK → CONCERNS on this head; the code blocker is closed (dispositioned above). Both remaining items are advisory.

  • WATCH / SUBTRACTION: the bash-script evaluator contradicts a documented invariant — replace it with an MCP tool on kirocrew-core and drop execute_bashaccepted-and-deferred to #5926

I checked the citation rather than taking it, and it holds — this is a sharper argument than the one I recorded when Design Review raised the same migration, and I had under-weighted it. docs/architecture/mcp.md "## The MCP-first rule" reads "A new LLM-facing capability MUST ship as an MCP tool, not only as a CLI command," and AGENTS.md:459 names exactly one deliberate exception (kirocrew computer call, "a human debug harness rather than a capability") with the explicit instruction "do not add another without reading docs/architecture/mcp.md." A bundled script the conductor invokes through execute_bash is an LLM-facing capability, so this PR does add a second de-facto exception, and it did so without that rule being weighed. Your second-order points are right too: the per-call approval prompt bounds nothing the script does not already bound, since after the cmd removal no spec field reaches the argv at all (the fixed ["gh","pr","checks",N] template is the boundary, pinned by test_no_spec_field_can_name_a_command) — so the prompt is pure friction, which is exactly the cost Design Review flagged from the other direction. The reason this is deferred rather than fixed is that it is a relocation of the capability, not a patch: the evaluator moves servers, _evaluate's dispatch is rewritten against the kind discriminator, execute_bash leaves the spec, and SKILL.md's invocation section and path-resolution paragraph go with it — a different PR's worth of change, on a file this PR has already spent four blocking rounds on and which the maintainer closed by formal override. I have added your citation to #5926 as the governing rule, so the migration now carries the reason it is mandatory rather than merely tidier, plus both consequences you named (execute_bash leaving the spec, the prompt disappearing).

  • WATCH / SUBTRACTION: the cmd tombstone has zero consumers — rebutted on proportionality; the evidence is accepted, the change is not taken here

Your grep is correct and my original justification for the branch was wrong: kind == "cmd" appears only in the script and its own test, no shipped skill ever emitted it, so the "older skill" consumer count really is 0 and the unknown-kind error verdict already refuses it safely. What holds me back is not the claim but the file. accept_eval.py took four consecutive blocking rounds this PR — interpreters on the allowlist, an argv[0] path bypass, an approved-wrapper bypass of the denied-command floor, then the planted-gh and file-probe pair — and the maintainer closed that span with a formal /ai-review override rather than a fifth patch. Deleting six lines of provably dead code re-arms every lane on the precise surface that was just settled, for zero behavioural change, and it is a strict subset of the rewrite #5926 performs on that same dispatch. So it rides with #5926 rather than being taken as a standalone edit here. Recorded as a deliberate deferral of a correct observation, not a disagreement with it — and this is the second round it has been raised, so it is the maintainer's call to overrule if the six lines are worth the round.

@iamwhatever

Copy link
Copy Markdown
Collaborator Author

Re-verdicted on this head; both items are advisory CONCERNS and both are the long-term shape of the role rather than defects in this diff.

  • WATCH: patrol blocks on a human approval every cycle, so the role trades reading N sessions for clicking N prompts — accepted-and-deferred to #5926

You have the security derivation right, and you have named the actual cost rather than a theoretical one: keeping execute_bash and @kirocrew-dashboard off allowedTools is correct because the deny floor cannot see inside an approved wrapper, and the price is that an unattended cycle waits. Your parenthetical is the fix — "an MCP verb rather than a shell wrapper" — and First Principles reached the same place on this head from the invariant side: docs/architecture/mcp.md's MCP-first rule plus AGENTS.md:459, which allows exactly one bash exception and says not to add another. An MCP tool on kirocrew-core is already mounted and auto-approvable in this spec, so the prompt disappears and execute_bash leaves the spec entirely. Both are now recorded in #5926 with that citation. It is deferred rather than fixed because it relocates the capability across servers and rewrites the evaluator's dispatch — not a change this PR can absorb — and your framing that the trade "should be validated before the role is promoted further" is the right gate: the built-in ships, the promotion waits on evidence.

  • WATCH: conductor runtime state is hand-serialized JSON inside the ledger's general-purpose artifacts map, with encoding and rotation enforced only by SKILL.md prose — accepted-and-deferred to #5912

Correct as stated, and the failure mode you name is the real one: the remedy in the skill's own text is "re-derive it from the child session", which is a recovery procedure, not a guarantee. What this PR could do inside its scope it has done — the docs now state the invariant the endpoint actually enforces (dashboard/handlers/session_ledger.py rejects a non-string value with artifacts_not_string_map, HTTP 400), and a ratchet test fails if that documentation drifts. What it cannot do here is give the item-entry format a code owner: encode, decode, validate and rotate would need to live in one module with types, which changes a shared ledger contract used beyond this agent. That is #5912, filed with the enforcement point named so whoever picks it up does not have to re-derive it. Shipping on prose is a real risk and I am not claiming otherwise; it is bounded by the fact that a malformed write loses one item's spec rather than corrupting the ledger.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Aug 25, 2026
@iamwhatever
iamwhatever disabled auto-merge August 25, 2026 21:56
@iamwhatever
iamwhatever merged commit 48cd4d6 into main Aug 25, 2026
34 of 38 checks passed
@iamwhatever
iamwhatever deleted the feat/conductor-builtin branch August 25, 2026 21:56
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants