feat(agents): add Agent Template authoring — create, edit, clone dialog + security predicates (#2255) - #2383
Conversation
c87f405 to
db9c14d
Compare
db9c14d to
a6868c6
Compare
|
Overlap notice — three PRs implement Agent Template authoring from the dashboard:
Recommendation: consolidate on #2383 as the base — widest scope (create/edit/clone + all #2255 items), the only branch without merge conflicts, and it matches what issues #1829 and #2255 asked for. However, #2383 is not yet merge-ready: CI is failing (frontend i18n gates, backend test shard, coverage gate), it ships no tests, and only the
@kyleseaman @RohanK6 — inviting you both to review #2383 so the best parts of all three land. If maintainers prefer the original sequencing (#2023 first, #2383 rebased back into the fast-follow it started as), that also works; the main thing is that the tests and i18n coverage from the other two PRs are not lost. |
|
I pushed a commit onto this branch ( Three of the fixes are load-bearing, and they share one root cause worth stating up front: kiro-cli validates 1. The prompt went to 2. There was also a mismatch inside the feature: the dialog sent the nested 3. PUT could overwrite Kiro Crew's own agent. The guard was Smaller ones in the same commit: PUT now carries forward spec keys the form doesn't model, so editing a description no longer deletes hand-authored Your On the red CI, both failures were fixable and are now green locally:
Verification on the new head: 32 endpoint tests + 55 predicate tests, all 7 schema fixes revert-verified (patched out, confirmed failing, restored). Backend 39,648 pass — the 22 remaining failures are host-environment (jq version, Two things I deliberately left alone: the branch is 82 behind main and CONFLICTING, and rebasing someone else's branch felt like yours to do — say the word and I'll take it. And one test asserts the written spec contains only keys kiro-cli accepts, so the next unknown-field slip fails there instead of silently disabling every template; if you'd rather that live somewhere else, move it. |
|
Resolved #2023 in favor of this one. Feel free to let me know if you'd like any help bringing anything worthwhile over! |
6b90d9e to
d130162
Compare
f9ba512 to
f80c1a9
Compare
|
Head is now GPT's five BLOCKING findings from
|
|
Head is now 1. Correct. This is the third round of findings on this one screen (traversal, then this), so it now enforces a rule instead of another special case: a Consequence worth stating plainly: 2. Both halves were real. A malformed spec was caught as 3. Real lost-update race. Existence check, read, carry-forward and write are now one serialized step under The test issues two genuinely disjoint PUTs concurrently (one touching only 4. Real data corruption: the args array was space-joined for display and re-split on whitespace at submit, so Advisory FINDING — Same answer as before, and it has not become less true: the Rebase note (worth a look, since it touched a file this PR did not own): main refactored the inline route table into Gates on |
|
Head is now GPT1. Verified before fixing, because a bogus restriction on 2. Correct, and it was my code. Worth noting how this one was verified, because the first attempt was a false pass: nothing in the suite distinguished the two readers, so reverting to 3. Real, and it was an interaction my own carry-forward created: 4. The whole entry shape is validated now ( Opus — the better catch of the round
This is the fourth instance of one class on this screen, and Opus did the work to prove it rather than assert it: Fixed by screening the relative form a second time with Advisory — three 400 bodies omit GPT's advisory FINDINGs
Gates on |
|
Head is now 1. Correct and one-line. 2. Two of the three parts were real and are fixed:
Fixing the transport rule invalidated three of my own earlier test fixtures that used args-only servers — the exact husk shape the rule now refuses. Two were updated to launchable entries. The third mattered more: On advertised-model validation: declined, and this is now the third round it has been raised, so here is the full reasoning rather than a one-liner. The dashboard's model picker and this endpoint share a source of truth ( 3. Correct, and it was the second site of the same defect: I routed the collision scan through 4. Real, and self-inflicted: the screens added in earlier rounds ( 5. Correct. Most refusals from the builder are security decisions — a sensitive resource or prompt path, a literal credential in an MCP env block, a wildcard auto-approve grant — and they returned 400 with no audit record. Both handlers now emit a Advisory — function-local imports — FIXED this round. Advisory — Gates on One local-only failure worth naming so it is not mistaken for a regression: |
|
Head is now 1. Correct, and it is the same audit-every-path gap one branch over: last round I added the denied event to the validation branch and left the managed-template 403 beside it silent. It now emits a 2. I declined this three times on the grounds that write-time refusal would be brittle when entitlement is momentarily unknowable. That objection does not survive reading the predicate: The check delegates to Doing it properly required one small refactor worth flagging since it touches a path this PR does not own: the provider walk that resolves what a live session advertises lived inline inside Three tests pin the new rule, including the fail-open case: an empty advertised set still accepts any model. 3. Real. Agent specs live in a user-writable directory shared with other tools, so a hand-edited Advisory — remaining function-local governance import — FIXED. My hoist script last round replaced only the first occurrence; a second lived in That hoist had a consequence worth recording: three governance tests patched Gates on On the shape of this review cycle. Nine rounds in, the finding count is falling (5 → 5 → 4 → 3) and this round produced no new defect class — two were adjacent instances of gaps I had half-fixed (audit one branch, hoist one occurrence), one was a genuine new input-validation case, and one was a finding I had wrongly declined. That reads like convergence rather than a treadmill. The recurring pattern in my own errors is worth naming plainly: when a fix applies to a class of call sites, I have repeatedly fixed the one named in the finding and left its siblings — the reader on the collision scan but not the update, the audit on validate but not managed, the hoist on one import but not two. Where a third instance of that shape shows up, the right response is to enumerate every site rather than patch the reported one. |
|
Head is now Opus — relative glob bypasses the credential-file screen (
|
|
Head is now MCP URL and argument credentials bypass screening (
|
|
🤖 Kiro Crew [operator: iamwhatever]: This PR has been inactive for 5+ days. I reviewed the blockers but they require your input:
When you've addressed these, the pipeline will re-assess on its next cycle. |
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
|
Rebased onto current main and pushed as The branch had fallen 679 commits behind, so this was a substantial re-anchor rather than a fast-forward. 14 conflicts:
The rebase surfaced a real defect worth calling out. Main's shared agent-name grammar ( Also translated the strings this branch adds — they were English in all 11 non-English catalogs, which the diff-scoped Gates on Prior rounds' fixes were revert-verified again after the rebase to confirm they survived it — the round-10 relative-glob HOME-base screen and the round-11 whole-entry MCP credential sweep both still fail when mutated out (3/3 caught). |
First Principles Review (Fable 5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS Every addition names its harm, but the dialog re-spells the model/skills writes PATCH What this change shipsIntent: let a user create, edit, and clone agent templates from the dashboard instead of hand-editing JSON — an ADDITION.
Watch
Subtractions
[FIRST-PRINCIPLES-REVIEWED] e5c57c6 |
|
Head is now GPT — new MCP rows always lack
|
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
|
Head is now GPT — PUT can overwrite concurrent external edits (
|
|
Head is now Worker thread races the loop-owned session map (
|
|
Head is now Removing a tool retained its auto-approval ( Dialog reset retained the previous MCP command ( Exclusive create published an incomplete file ( Two of my three tests were false passes on the first attempt, worth recording since the corrections are the interesting part:
My first two mutations were also aimed wrongly (one landed after staging had already happened), so the harness reported false passes on correct code until the mutation targeted which implementation the call site uses. Gates on |
|
👋 Hi! This PR's description is missing some required sections from our PR template. Workflow runs won't be auto-approved until the description is updated. Missing sections:
Please update your PR description to include these sections, then push or re-save the description. The workflows will be approved on the next cycle. |
|
Head is now Template mutations lack owner authorization (
|
|
Head is now Unchanged stale fields overwrite newer edits (
|
…odotdev#2255) Adds dashboard authoring for custom agent templates so a user can create, edit and clone `~/.kiro/agents/<name>.json` instead of hand-editing JSON. Backend - POST /api/agents/installed — create a user-owned template (validates name charset, description, model, prompt, tools, allowedTools, and mcpServers with credential screening on env blocks plus path-sensitivity on command). - PUT /api/agents/installed/{name} — replace an existing user-owned template. - Shared credential predicates in security.py: env_key_is_credential_like() with token-split matching, MCP_ENV_SECRET_VALUE_RE for known secret shapes, and an anchored ${VAR} reference form as the sanctioned escape. Frontend - AgentTemplateCreator.tsx — authoring dialog with create/edit/clone modes, tool chips with per-tool auto-approve, inline MCP server rows, skill catalog toggles, and draft-commit on submit. - AgentsPage.tsx — Create in the roster header, Edit/Clone in the inspector for user-owned templates only. Spec correctness kiro-cli validates these files with serde `deny_unknown_fields` and rejects the ENTIRE spec on any unknown key, then silently falls back to the default agent. An unknown field is therefore not a degraded template but a template that does not exist, while the session appears to run the user's agent. Three consequences shaped the field set: - The prompt is written to `prompt`. `customInstructions` exists nowhere in this codebase and would have made every template carrying a prompt unloadable. The body alias is still accepted; only the written key differs. - `deniedCommands` is refused rather than stored. Top-level it is an unknown key; under toolsSettings.execute_bash it would revive a retired mechanism that agent.py:_strip_legacy_denied_commands deletes on every refresh, so a rule authored here would either vanish or shadow Settings > Security. - PUT refuses OWNED_KIRO_AGENT_FILES, not just `<app>--<agent>.json`. No managed filename contains a double dash, so a `--`-only guard let a PUT full-replace Kiro Crew's own spec and drop hooks (including the bash audit hook), includeMcpJson and the managed MCP block. Durability - Create uses O_EXCL rather than exists() then os.replace, so a concurrent POST cannot silently clobber the loser. - Create rejects a name any existing spec already answers to via its `name` field; kiro-cli resolves by that field, so a free filename is not enough. - Managed stems and the built-in `default` are reserved. - PUT carries forward spec keys the form does not model, so editing a description no longer deletes hand-authored hooks or toolsSettings. Tests - 32 covering both endpoints, including one asserting the written spec holds only keys kiro-cli accepts, so the next unknown-field mistake fails there instead of silently disabling every template. - 55 covering the credential predicates in both directions: a false negative writes a secret into a 0644 spec, a false positive blocks legitimate config and teaches users to route around the screen. - Every schema and durability guard revert-verified. Co-authored-by: Kyle Seaman <kseam@amazon.com>
|
Scope coordination note with #5161: both branches edit the agent mutators in Please preserve the authoring and security-predicate behavior here, but rebase onto the shared owner-helper baseline and avoid carrying a second copy of the agent mutation gates already merged via #5011/#6245. That keeps template semantics separate from the residual authorization sweep. |
Open PR relationship auditThis is a consolidated, point-in-time code-level audit note. It compares complete merge-base diffs and current/merged code; it does not treat a shared topic as duplication or partial coverage as completion. Relationship findings
No PR, Issue, label, branch, or review state was changed by the relationship-note portion of this audit. |
|
@qh2244 the 2026-09-08 open-PR audit found two open PRs that touch the same code as this one. #8307 (@xuejinT) — same feature, incompatible design. Shared: #6307 (@kyleseaman) — one real conflict. Both rewrite the same lines of Posted from the 2026-09-08 open-PR relationship audit (read-only, one auditor per PR); reply here if any of this is wrong. |
Problem / Motivation
Custom agent templates could only be created by hand-editing JSON under
~/.kiro/crew/agents/. The dashboard listed templates and let you pick one, but offered no way to author or change one — so a user who wanted an agent with a particular model, prompt, tool set, or MCP server had to leave the UI, find the directory, learn the schema, and get the file right by hand. A malformed field there is not reported in the UI: kiro-cli rejects the whole spec and the session silently falls back to the default agent.Why it matters
Authoring an agent is the main way a user shapes KiroCrew to their own work, and it was the one part of the roster the dashboard could not do. The JSON route also puts the burden of the security-relevant fields on the user:
allowedToolsgrants auto-approval,mcpServers.*.envis where a literal API token ends up pasted, andresourcesglobs can be pointed at~/.awsor~/.ssh. Doing this in the UI is what makes those fields screenable at a single chokepoint rather than trusted because a human typed them.What changed (motivation → approach → change)
Goal: create, edit, and clone agent templates from the roster, with the schema and the security rules enforced server-side rather than assumed of the author.
Approach. Two endpoints (
POST /api/agents/installed,PUT /api/agents/installed/{name}) behind one validation path, plus a structured dialog that never asks the user to type JSON. Two design decisions carried most of the weight:_persist_spec,_screen_both_bases, a whole-entry MCP sweep) rather than at each endpoint, because the recurring bug class in review was a rule applied to the site a finding named while its siblings kept the hole.What was built. Backend: the two endpoints with owner authorization, SEL-audited denials, name validation against both the shared agent grammar and the template grammar, model validation against advertised ids, credential screening over whole MCP entries (
env,headers,url,args,command), path-sensitivity screening ofresourcesglobs against both the project and HOME bases, an exactly-one-transport rule per MCP server, and an optimistic-concurrency check that returns409 agent_template_conflictrather than losing a write. Writes stage to a temp file and publish withos.link, so a create is atomic and fails if the name already exists. Frontend:AgentTemplateCreator.tsx(create/edit/clone, tool chips with per-tool auto-approve, MCP rows, skill catalog), Create in the roster header, Edit/Clone in the inspector, andagentCreate/agentUpdateinclient.ts. Security predicates that both sides need live inkiro_crew/security.py.Tests
test/test_agent_template_authoring.py— 190 tests. The write surface end to end: validation per field, absent-means-preserve on update, the409conflict path and the version token's sensitivity to a same-size same-mtime change, owner authorization on both endpoints, atomic publish and the refusal to publish truncated JSON when hard links are unavailable, credential screening scoped to values a request changes, relative-glob traversal against both bases, and the exactly-one-MCP-transport rule.test/test_mcp_env_credential_screen.py— 84 tests. The credential predicates: token-split key matching, literal-value shapes,${VAR}reference allowance, and the metadata-suffix exemptions.website/src/test/AgentTemplateSubmitPayload.test.tsx— 17 tests. What the dialog actually sends: unchanged fields omitted, changed fields sent, a cleared field sent (clearing is a change) while a never-loaded empty field stays omitted, unmodelled MCP fields and space-containing argv surviving a round trip, and a clone carryingresourcesexplicitly because POST has nothing to preserve against.Each fix in review was revert-verified — the fix patched back out to confirm the paired test fails — because several tests initially passed for the wrong reason (an exception the old path also raised; a remount that reset the state under test).
Manual verification
Pod screenshot verification of the create, edit, and clone flows (below). The screenshots need a refresh: they predate the MCP transport field and the 320px stacking added during review, so they show the dialog one revision behind. Re-capturing before merge.
Screenshots
Inspector with Edit / Clone buttons (user-owned template selected)
Edit dialog — name disabled, fields pre-filled from existing template
Clone dialog — name cleared, rest pre-filled
Related Issues
Closes #2255. Subsumes the scope of #2023. Supersedes #2148, which was closed in favour of this branch.
Review of this PR surfaced a class of pre-existing unguarded mutating endpoints beyond its scope; enumerated in #4944 rather than rewritten here.
Open questions for the reviewer
Three UX items are deliberately left as product decisions rather than being decided in a review round:
!managedgate hides Edit and Clone together, while the backend's 403 text tells the user to "clone it to get an editable copy".Checklist