refactor: move backend capability tables into agent_sdk (PR 3a) - #9381
Conversation
|
Intent: Move the ACP backend registry and the tool-gate routing verdict behind the agent-SDK boundary (RFC PR 3, option 1), and make the six consumers that still asked which backend a session is on ask what the backend can do instead — with every answer, for every backend id and for an unknown one, identical to Not a goal: the rest of RFC PR 3 (the |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Capability-over-identity is the right cure for H6; behavior equality is held by literals pinned from Suggestions
[DESIGN-REVIEWED] 4ae969c |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The non-deprecated shims keep the old import path a new consumer finds first — the exact failure the PR's rationale rejected — plus two zero-consumer surfaces. Not justified as shipped
What this change shipsIntent: stop application code from branching on which backend a session runs, so a new harness cannot silently inherit an arm it never demonstrated — an ADDITION (structural refactor executing the repo's recorded RFC decision).
Watch
Subtractions
[FIRST-PRINCIPLES-REVIEWED] 4ae969c |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
bdc961e to
8c8ceef
Compare
|
span=9ec07ed2ded9
self-added: no
Pre-drafted override rationale for a maintainer, verified independently against the code |
|
span=25513d154ea8
self-added: no
Pre-drafted override rationale for a maintainer, verified independently against the code |
8c8ceef to
9e66635
Compare
Blocked on a pre-existing
|
| file | markers removed |
|---|---|
src/kiro_crew/agent_sdk/backends.py |
4 |
src/kiro_crew/agent_sdk/capabilities.py |
1 |
src/kiro_crew/agent_sdk/tool_gate.py |
2 |
src/kiro_crew/acp_backends.py |
1 |
test/test_agent_sdk_capabilities.py |
7 |
test/metrics/test_turn_profile.py |
1 |
test/test_subagent_coverage.py |
1 |
I also ran check_comment_history.py --write-baseline as the gate asked, which pruned
the two entries the module move emptied (acp_backends.py 3 → 0 and acp_tool_gate.py
2 → 0) and lowered _total from 7606 to 7601.
Why I am not "fixing" the remaining six
The two available moves both look wrong to me, so this is a maintainer call:
- Delete six markers from
chat_runner.py. They are(#2696 GPT round, blocking)/
#2686references inside the promise-only-turn recovery, and each records why a
specific guard exists — which reviewer demanded it and which symptom it prevents.
The rule targets comments narrating what code used to do; these narrate why a
guard is there, which is the WHYcode-style.mdasks comments to carry. Removing
them to satisfy a count would delete the most useful thing in that block, in a
mechanism this PR does not otherwise touch. - Raise the entry 103 → 109. The baseline's own header forbids it: "Do NOT add or
raise an entry to make a red gate green."--write-baselineenforces that — it
never adds or raises — so the tool cannot record this correction either.
That leaves a real gap in the ratchet worth naming: a PR that lands new markers and
runs --write-baseline ships with an entry that silently understates its file, and the
next person to touch that file inherits the red. That is what happened to
chat_runner.py.
What would unblock this
Either is fine by me; both are yours to choose:
- Correct the
chat_runner.pyentry to the tree's real count (109) in its own commit, so
the correction is reviewable as a baseline fix rather than hidden inside a refactor; or - tell me to pay the six markers down in
chat_runner.py, and I will rewrite each into
present tense while keeping the guard's reason (naming the symptom instead of the issue
number).
Everything else on this PR is green or answered: GPT's round-1 blockers are gone (its
current stamp carries one advisory FINDING), Opus reports no findings, and both round-1
dispositions are posted with no disposition-rule violations.
|
span=d02faec46615
self-added: yes
|
9e66635 to
7fab54b
Compare
|
Attribution for head |
10523da to
735d34c
Compare
735d34c to
6009f26
Compare
6009f26 to
0006b1b
Compare
Consolidates the ACP backend registry and the tool-gate routing verdict behind the agent-SDK boundary, and replaces the six backend-identity checks that still lived outside it with capability questions. RFC PR 3, option 1. acp_backends.py -> agent_sdk/backends.py and acp_tool_gate.py -> agent_sdk/tool_gate.py. Both top-level modules stay as pure re-export shims, so no existing call site changed in the same commit as the move. New agent_sdk/capabilities.py carries SessionCapabilities, one field per question a consumer outside the boundary asks; every field translates a table that already existed. scripts/check_harness_parity.py's VOCABULARY_PATH follows the vocabulary to its new home. No behaviour change: every routing verdict, permission config, membership answer and capability field is pinned to a literal copied from a clean main checkout, for every backend id and for an unknown one.
0006b1b to
4ae969c
Compare
Problem / Motivation
Six places outside the agent-SDK boundary decide what to do by asking which
backend a session is on, not what that backend can do:
config/loader.pycomparesagent.acp_backend == ACP_BACKEND_CLAUDE.dashboard/chat_handlers.pyanddashboard/handlers/agents.pyreadprovider.is_claude_backend.dashboard/chat_runner.pydoes it three times: the pinned-model verdict, thebilling row's provider label, and the
/compactbranch.knowledge/llm_pool.pyreadsAcpClient._is_claude— a private attributeof a class in another package.
subagent.pylazily importsproviders.acp.is_claude_backendto pick whichhome tree to clean up.
The tables those checks are really about also sit outside the boundary:
acp_backends.pyowns the backend ids, 16ACP_BACKENDS_*capability sets, theRoutingenum and the two dispatch tables;acp_tool_gate.pyowns thePreToolUse routing verdict and the adapter credential mask.
Why it matters
An identity check has one failure mode and it always points the same way. Add a
fifth backend, edit nothing, and it silently takes whichever arm "not claude"
selects — an arm it never demonstrated it can serve. That is the failure the
harness-parity doc calls H6, and
codexis the harness that just landed.The
_is_clauderead is worse than the others. It is application code reachingan underscore attribute of the ACP client, so the client cannot rename its own
private field without breaking the knowledge pool.
Leaving the tables outside also means the boundary is advertised rather than
real: the old import path still works, and a new consumer finds it first.
What changed (motivation → approach → change)
Each branch depends on a property of the backend. The property already had a
table. So the tables moved behind the boundary, and each branch now reads the
property.
acp_backends.pybecameagent_sdk/backends.py, andacp_tool_gate.pybecameagent_sdk/tool_gate.py. Both old modules stay as pure re-export shims, so about30 existing call sites needed no edit. The shims re-export the functions, not
copies, so the registry still has one
_baseline/_selectablepair whicheverpath imported it.
New
agent_sdk/capabilities.pyholdsSessionCapabilities, a frozen record withone field per question a consumer actually asks, plus
capabilities_for(backend)and
capabilities_of(provider).AcpProvider.capabilitiesis where a livesession's record comes from. Every field translates a table that already existed,
so no membership changed and no backend changed arms.
flowchart LR subgraph Before C1[consumer]:::ctx -->|"is this the claude backend?"| P1[provider / client]:::removed C1 -->|"reads a set directly"| T1[acp_backends.py]:::removed end subgraph After C2[consumer]:::ctx -->|"what can this backend do?"| K[SessionCapabilities]:::added K --> T2[agent_sdk/backends.py]:::added S[acp_backends.py shim]:::changed -.->|re-export| T2 end classDef added fill:#DCFCE7,stroke:#16A34A,color:#14532D,stroke-width:2px classDef changed fill:#FEF3C7,stroke:#D97706,color:#78350F,stroke-width:2px classDef removed fill:#FEE2E2,stroke:#DC2626,color:#7F1D1D,stroke-dasharray:4 3 classDef ctx fill:#E0F2FE,stroke:#0284C7,color:#0C4A6E linkStyle 0,1 stroke:#DC2626,stroke-dasharray:4 3 linkStyle 2,3 stroke:#16A34A,stroke-width:2px🟩 added · 🟨 changed · 🟥 removed · 🟦 unchanged
A consumer now asks what the backend can do; the tables it asks about live behind
the boundary, and the old module path still answers.
The six sites and the field each one reads:
config/loader.pymodel_id_namespacedashboard/chat_handlers.pymodel_id_namespacedashboard/handlers/agents.pyresolves_model_from_advertised_listdashboard/chat_runner.py(pin verdict)resolves_model_from_advertised_listdashboard/chat_runner.py(billing row)provider_seamdashboard/chat_runner.py(/compact)compacts_inlineknowledge/llm_pool.pyeffort_via_config_optionsubagent.pyprovider_seamOne capability set is new.
ACP_BACKENDS_INLINE_COMPACTIONexists because the/compactbranch was the one question with no table behind it. Its membership isexactly what the check it replaced answered, and it is a strict subset of
ACP_BACKENDS_COMPACT.Three more things the move required:
ACP_BACKENDS_*set now has a recorded disposition inagent_sdk/backends.py's docstring — semantic question, pre-session registryquery, or driver-internal — and a test fails if a set has no row.
scripts/check_harness_parity.py'sVOCABULARY_PATHfollows the vocabulary toits new home. Pointing it at the shim would have been the drift the H8 rule
exists to prevent: a second place a definition could legally live.
importing
providers.acpat all:dashboard/chat_runner.py5 → 4 andsubagent.py8 → 7. Nothing was added or raised.RFC
docs/request-for-change/rfc-crew-agent-sdk-boundary.mdasked PR 3 to pickone of three homes for
Routingand the dispatch tables and record the choice.This lands option 1 and records it, plus a "PR 3a LANDED" note saying what is
still PR 3's to do (the role protocols,
SessionRequest.mcp_servers,writes_own_transcripts,spawnable_multiplexed_selections()).Zero behaviour change, and how that is held
Every routing verdict, permission config, membership answer and capability field
is pinned in
test_agent_sdk_capabilities.pyto a literal copied from a cleanmaincheckout, for every backend id and for an unknown one:""(kiro)agent_spec("", "")kasagent_spec("", "")claudeseeded_settings("", "")codexsession_config("mode", "read-only")unverified("", "")"kiro"(the policy wire name, not a backend id)unverified("", "")capabilities_ofkeeps the calling convention the six predicates had. They wereisinstance(provider, AcpProvider) and provider.is_claude_backend, so a shapethat was not a provider answered False; it requires a real
SessionCapabilitiesinstance, so a wrapper, an unstarted provider or a
MagicMock(spec=...)lands onthe fail-closed default instead of claiming every capability at once.
One residue is preserved deliberately:
provider_seamstill labels a KAS turnacp. Only the ACP layer's ownPROVIDER_LABEL_*constants separate it, andpromoting that would change what every KAS turn records — a telemetry change, not
a refactor.
One identity read stays
dashboard/handlers/agents.py'sapi_modelsstill compares againstACP_BACKEND_CLAUDE. It asks whether the backend has a--list-modelscatalog toshell out to, which is a pre-session question whose capability answer would have
to be decided for KAS and Codex rather than translated. It is pinned by
enclosing function name, so a seventh read cannot appear beside it.
Tests
New
test/test_agent_sdk_capabilities.py(37 tests):naming a backend identity, with the one remaining read allowed by enclosing
function and a second test that fails if that function stops reading one.
capabilities_for(...),capabilities_of(...)or.capabilities, so one grep finds every consumer;plus every field has at least one consumer, and the field list cannot drift from
the dataclass.
__all__), so the boundary has no second frontdoor.
above, and the fail-closed answer for an unknown id.
ACP_BACKENDS_*set has a disposition row, and no row names a set or afield that does not exist.
ACP_BACKENDS_INLINE_COMPACTIONis a strict subset ofACP_BACKENDS_COMPACT.AcpWorkernever passesacp_backend, which is why swapping_is_claudeforthe effort capability changed nothing; if a future pool selects a backend, this
fails instead of inheriting an arm.
Updated tests, all for the same reason — a double that asserted an identity flag
now carries the real capability record for the backend it stands in for:
test_cc_models_endpoint,test_dashboard_chat,test_dashboard_chat_handlers_coverage,test_acp_model_config_options,test_llm_pool,test_subagent_coverage,test/metrics/test_turn_profile. The three that restore the private registry pair(
test_harness_parity,test_agent_backend_editable,test_agent_backend_governance)now reach
agent_sdk.backends, which defines it.test_acp_capability_sets_leafand
test_agent_sdk_backend_identityfollow the vocabulary to its new module andadditionally assert the shim is a shim.
test_subagent_coverage's_is_cc_providertests gained the case that used to bemissing: a shape that is not a provider must answer False.
Manual verification
N/A — unit coverage sufficient. This is a module move plus predicate translation
with no new user-facing surface, and the no-behaviour-change claim is held by
literals copied from
mainrather than by inspection.Local verification, for the record:
prepare-prprofile: green.check_agent_sdk_boundary.py: green, baseline lowered by 2.origin/mainworktree: 270 failures fail identically on the base (this host has no user
namespaces, so the sandbox, pod, trash and session-storage suites are red
regardless), and zero are caused by this diff. One that first looked
diff-caused (
test_security_conductor_skill_contract) was fixed onmaininfix(test): security-conductor scripts guard checks for stubs, not absence #9362 and went green on rebase.
docs_lint: green. Its one report-only stale baseline entry(
line-ref … acp/client.py:3313) reproduces onmainand is not from this diff.Related Issues
no linked issue: this is a step of an RFC that has no tracking issue; the design
of record is
docs/request-for-change/rfc-crew-agent-sdk-boundary.md, PR 3.Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)