feat(acp): probe backend install state and gate selection by policy

[iamwhatever]

Problem / Motivation

Developer → Agent Backend cannot tell a user why a backend is unavailable, and its
one signal is the wrong kind of fact.

AgentBackendTab derives every option's disabled state from GET /api/config/schema
enumValues, which resolves from acp_backends.selectable_backend_values() — a
build/edition fact. Nothing checks whether the harness's components are installed
on this machine, so:

• A user with Claude Code's CLI installed but not the claude-agent-acp adapter sees
  a dead option and no reason. The resolvers that could answer exist
  (_resolve_claude_acp_bin, _resolve_claude_code_executable) but are called only
  at spawn time and never surfaced.
• A managed fleet cannot bound which harness it runs. Every other governable surface
  has a SCOPE_CATALOG row (capabilities.publish, yolo_duration, …); the agent
  backend has none, so "only run the harness we qualified" is unexpressible
  (tracked as Add an agent_backend governance scope to bound which ACP harness a fleet may select #6622, which the selectable-backend registry from feat(config): one registry for selectable ACP backends instead of three literals #6593 left open).

Why it matters

The first is a dead end a user cannot diagnose: the panel says nothing, and the two
components have different remedies, so a bare "unavailable" sends someone after the
half they already have. The second means an enterprise that qualified one harness has
no way to stop a user selecting another.

What changed (motivation → approach → change)

Two independent facts decide whether a harness can be selected, and this PR adds the
two that were missing — deliberately at different layers.

1. Machine probe → whether the option can be clicked.

New agent_sdk/backend_install.py answers per backend through the same resolvers the spawn
uses, never a reimplementation — a probe that agreed with a hand-rolled PATH search
but disagreed with the spawn would tell an operator they are ready and then fail the
session. Three states, and the third is load-bearing: unknown means the check itself
raised, and is never collapsed to missing, because that would send someone to install
what they may already have. Claude needs both components and names the absent one,
so the half-install case gets an actionable line plus the adapter's npm command (read
from CLAUDE_ACP_NPM_PKG, not restated). KAS shares kiro's answer structurally: it is
not an independent harness — kas_transport.build_kas_argv returns
[kiro_bin, "acp", "--agent-engine", "v3", …] — so there is no separate resolver to
disagree with. Served owner-only at GET /api/acp-backends, offloaded via
asyncio.to_thread because the resolvers shell out, with a short TTL cache because the
dashboard polls.

AgentBackendTab disables on !selectable || installed === 'missing' || restart_required
and fails open everywhere else: unknown, the query in flight, a 403 (non-owner) or
a 404 (older gateway) all leave the option exactly as it was, since the PATCH allowlist
is the real gate and an optimistic disable costs a user a control they were entitled to.

2. agent_backend governance scope → whether the deployment may select it.

Semantics is the decision #6622 was blocked on: additive over a floor.
{"agent_backend": {"mode": "allow", "allow": ["claude"]}} means also allow claude.
GOVERNANCE_FLOOR_BACKEND (kiro) is never submitted to the scope, so no rule can
remove it — the exclusive reading can empty the set, and an install with no startable
harness cannot be repaired from the dashboard, since the trust-root policy is the one
file the dashboard may not write. Members are policy ids (kiro/kas/claude)
because the kiro backend is the empty string internally, which no identifier matcher
can carry.

Enforced by recomputing the registry, not by a check anywhere downstream —
narrow_selectable_backends() from bootstrap_context at gateway start, via a new
apply_selectable_denials() that assigns baseline - denied. That position is forced
by three harness-parity invariants, and the obvious alternatives each break one:

Candidate position	Broken invariant
inside resolve_selected_backend (the one gate)	H3 — it runs inside KiroCrewConfig.load(); resolving a ceiling loads config and re-enters that load
in create_provider_factory	H13 — names that function and forbids the Kiro construction path gaining a conditional in service of an adapter
intersected in the dashboard handler	H4 — selectability has exactly ONE gate; a second derivation is the drift the registry replaced

Narrowing at boot satisfies all three: the context is installed (no re-entrant load),
the registry stays the single source, and no call site changes. The existing single gate
resolve_selected_backend reads selectable_backends() per call, so it degrades a
policy-denied persisted value to the floor with a logged reason for free — which also
closes the "config.json written before the policy arrived" case with no new code.
The scope is asked with HOST_SESSION_KEY, not an empty key: an empty key classifies
to surface unknown and matches no profile, so a host-bound profile would be silently
ignored. Every decision is SEL-audited in both directions, which is cheap because this
runs once per gateway start.

What the card's last line means. The set of harnesses offered is decided when the
gateway starts, and editing the policy takes effect on the next start. The panel also
cannot detect a policy file edited by hand and not yet distributed — nothing in the UI
can, because seeing it would mean reading the trust-root policy on a request path,
which H3/H13 forbid. Stating the semantics is the honest substitute for a freshness
signal the panel is not allowed to have.

When a policy change binds, and why that is a decision. The set is recomputed at
gateway start only. apply_ceiling does replace current_context().governance
mid-process — policy_distribution's poll thread reaches it through refresh_now() —
and every other scope picks that up on its next decision. This one deliberately does
not.

An earlier revision of this branch DID re-derive the registry there. Re-deriving binds
the new ceiling for backend SELECTION and nothing else: sessions already running the
denied harness, and providers already in the warm pool, keep going, because retiring
live work is a session-lifecycle capability that does not exist yet. The
operator-visible result is the worst of both — the option vanishes from the panel while
the harness is still in use, which reads as "it stopped being used". So the promise is
the narrow one that can be kept, and it is stated rather than implied: the panel says
the set is decided at gateway start and that editing policy takes effect on the next
start, and test_apply_ceiling_does_not_renarrow_the_registry pins the omission so it
cannot be mistaken for an oversight and quietly closed without the retirement half.

The recompute primitive is still non-destructive:
apply_selectable_denials(denied) ASSIGNS baseline - denied rather than mutating,
keeping the edition-registered baseline apart from the effective set. That is
what Design Review asked for, it makes the boot pass idempotent and order-independent,
and it means adding the runtime call site once retirement exists is a one-line change
instead of a redesign. narrow_selectable_backends() iterates the baseline, never the
already-narrowed set, because asking the narrowed set what to narrow is how a one-way
ratchet gets built by accident.

Probe vs spawn cache. _claude_acp_argv_cache is a process-lifetime cache behind an
_UNRESOLVED sentinel, never invalidated, so a fresh probe could report installed
while every spawn reused a cached None and died with AcpError — the exact
"told you it was ready, then failed" trap the probe exists to prevent. The probe
therefore consults that cache rather than bypassing or invalidating it (invalidating
would make a dashboard GET mutate a spawn-path global) and reports
installed + restart_required, which disables the option and says so. The opposite
skew (cached positive, adapter since removed) needs no special case: the fresh resolve
reports missing, which is what the spawn will effectively be.

Where the probe lives, and why it is not a free choice. The probe must reach the
spawn's own argv resolvers, and scripts/check_agent_sdk_boundary.py forbids
application code from importing kiro_crew.acp — a baselined gate whose own docstring
says never raise a baseline line to make it green. So the probe sits inside the
boundary, in the shape the RFC prescribes rather than one I picked:
docs/request-for-change/rfc-crew-agent-sdk-boundary.md §5.5 puts argv resolution
inside the driver and names agent_sdk/drivers/acp as the only module permitted to
import kiro_crew.acp, and §6.1 bucket 8 ("a second native binary the adapter's own
SDK will not find") is literally the claude-agent-acp + claude pair being probed.
agent_sdk/drivers/acp.py therefore owns the resolver calls and returns plain data;
agent_sdk/backend_install.py owns the states, components, TTL cache and per-backend
registry and imports no ACP; agent_sdk/__init__.py exports the capability. This makes
the install probe the SDK's first tenant — the package was declared empty on purpose,
and the boundary it names is what decided the placement.

Relationship to #6307. That open PR contains a _probe_installed of similar shape.
This implements independently on merged main — #6307 restructures acp_backends.py
into acp/backends.py and holds its admitted set at kiro+kas behind documented release
blockers, so depending on it would block this indefinitely. The probe's shape (three
states, spawn-resolver parity, owner-only endpoint) deliberately mirrors it so a later
merge collapses cleanly.

Tests

• test/test_agent_backend_governance.py (28) — the scope row and the kiro wire
  spelling; additive-allow (an allow list that would be exclusive under the other
  reading); an empty allow still leaving the floor; deny removing a non-floor backend
  and being inert on the floor; fail-closed leaving exactly the floor; narrowing never
  raising into boot. The registry recompute has its own class: denials are assigned
  not subtracted (so a looser second pass RESTORES what a stricter first pass removed),
  the recompute is idempotent, the floor is force-kept even when a caller names it,
  registering writes the baseline too, and registered_backends() stays the
  pre-narrowing answer. When a policy change binds has three: a ceiling installed
  mid-process does NOT re-derive the set (the contract, with the comment naming what
  else must land before that changes), the ceiling itself is still installed for every
  scope that reads it per decision, and the same denied policy taken at boot does bind.
  Plus the
  enforcement position: the dashboard allowlist deriving only from the registry, the
  single gate degrading a denied persisted value, and two source-shape assertions — that
  create_provider_factory contains no governance call (H13) and that acp_backends
  neither imports kiro_crew.platform nor calls current_context (H3). The latter is
  parsed with ast, not grepped: that module's docstring discusses current_context()
  at length to explain why it must not reach it, so a substring assertion would fail on
  the documentation while a real call added inside a function would pass once the prose
  was reworded.
• Host identity is pinned end-to-end: a profile bound to surface: host denying kas
  must actually remove it. With an empty session key that test fails.
• test/test_agent_sdk_backend_install.py — installed/missing/unknown per backend; a
  raising resolver yielding unknown and never missing; the specific absent claude
  component; kas tracking kiro; TTL cache used and clearable; owner-gate refusal and the
  payload shape. Six cover the spawn-cache divergence, including three false-positive
  guards (the _UNRESOLVED sentinel and a malformed cache value are not negatives — a
  restart prompt shown to every freshly started gateway would make the flag ignorable)
  and one pinning that the probe does not mutate the cache.
• website/src/test/AgentBackendTab.test.tsx (25) — missing disables and renders
  the command; unknown stays ENABLED and does not read as missing; a 403 and an
  in-flight query behave exactly as before (no disabled flash); not-selectable wins over
  any install verdict and over the restart line; the restart case disables and explains;
  and a selectable+installed backend reads exactly as it did before. One test builds its
  client with the app's REAL staleTime: Infinity rather than the convenient default,
  because that is the only shape in which the refetch gap is visible at all.
• Every behaviour this round changed was mutation-checked: hardcoding
  restart_required=False, swapping HOST_SESSION_KEY for "", and dropping the
  probe query's refetchInterval each redden their own tests and nothing else.

Manual verification

• Called the probe unmocked on a macOS host: kiro and kas → installed; claude →
  missing(["claude-agent-acp"]). That host has the claude CLI
  (_resolve_claude_code_executable resolves it) and not the adapter — the exact
  half-install the two-component split exists for.
• Verified the flip on a real resolution path rather than a mock: pointing
  CLAUDE_AGENT_ACP_BIN (step 1 of _resolve_claude_acp_bin, which the spawn honours
  identically) at a real file turns the verdict to installed with
  missing_components empty — without touching the machine's global npm environment.
• Frames below captured with scripted Playwright (website/scripts/capture-agent-backend-probe.mjs,
  committed), serving the real built bundle with fixture APIs. The first scene is this
  host's actual answer; the others vary one field, because installed: "unknown" only
  happens when the probe itself raises and a public build never reports claude as
  selectable — so shooting only local truth would leave the lines this PR exists for
  undocumented.

Screenshots / video

The two states this change exists to distinguish. Claude Code is disabled in both, for
different reasons, and the wording says which:

The check-failed state, and this host's real answer

unknown leaves the option enabled and must not read as missing — the check failed,
which is not evidence the binary is absent:

This host's actual answer on a public build — claude is not selectable at all, so the
build reason outranks the machine reason:

Related Issues

no linked issue: this closes nothing on merge, deliberately. It settles the semantics
question in #6622 (the agent_backend scope and the allow-is-additive ruling) and
enforces the scope at the registry, but that issue also asks for enforcement at every
chokepoint, so it stays open.

Related: #6593

Checklist

• At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
• Existing tests pass and new tests added for new functionality
• Self-review completed; code follows project style guidelines
• Documentation updated (if applicable)
• No secrets, credentials, or internal references in the diff

[github-actions]

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 68af3f7218072fa27f9daafe5bce597c1998b705 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 68af3f7

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 68af3f7218072fa27f9daafe5bce597c1998b705: <one-sentence reason>

[github-actions]

UX Review (Fable 5) — ✅ PASS

UX-level review of 68af3f7218072fa27f9daafe5bce597c1998b705 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All evidence gathered. The product does have an existing gateway-restart control (RestartButton, restart_gateway strings), which informs one finding. Emitting the review.

UX-Verdict: PASS

Every probe verdict renders as a plain, actionable sentence in the row it gates, fail-open states stay muted, and the screenshots confirm all four states read cleanly.

Suggestions

• installed_restart_required ends at "must restart before it can be used" with no restart path on the tab, while the product already ships a one-click gateway restart (RestartButton, pages…restart_gateway) — co-locate that action or say "restart the gateway" to match sibling wording.
• missing_components_with_command embeds npm i -g @agentclientprotocol/claude-agent-acp as inline warn-colored prose (agent-backend-missing.png) — render the command in monospace with a copy affordance so the one thing the user must transcribe exactly isn't hand-selected from a sentence.

[UX-REVIEWED] 68af3f7

[github-actions]

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 68af3f7218072fa27f9daafe5bce597c1998b705 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Sound layering and honest failure semantics, but the shipped docs contradict themselves about when a fleet policy binds — half still describe a reverted runtime call site.

Watch

• Stale "runtime ceiling install" claims survive an intra-branch revert. The PR deliberately does not re-narrow on apply_ceiling ("the selectable-ACP-backend set is deliberately NOT recomputed here", pinned by test_apply_ceiling_does_not_renarrow_the_registry), yet four places still assert the removed call site exists: narrow_selectable_backends's docstring ("Called from BOTH bootstrap_context … and policy_distribution.apply_ceiling"), apply_selectable_denials's docstring ("driven from … AND from policy_distribution.apply_ceiling whenever a ceiling is installed at runtime"), the SCOPE_CATALOG comment ("on every runtime ceiling install"), and governance.md's summary line ("at boot and on every runtime ceiling install") — which its own detailed section then contradicts. A maintainer or fleet operator reading any of the stale copies concludes a pushed deny binds immediately when it binds only at next gateway start — documentation claiming stronger enforcement than exists, on the enterprise-admin boundary. Fix: align all four to the boot-only contract in the same commit.
• The boot-only binding window is unbounded and silent on the admin side. Every other scope picks up a pushed ceiling on its next decision; this one leaves a newly denied harness selectable for new sessions until each gateway happens to restart, and nothing tells the operator which gateways are still on the old set. The tradeoff is argued, documented, and test-pinned, but a human should ratify accepting an indefinite enforcement lag over the "new selections bound, live sessions grandfathered" model the rest of governance uses.

[DESIGN-REVIEWED] 68af3f7

[github-actions]

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 68af3f7218072fa27f9daafe5bce597c1998b705 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Analysis complete. The change is two declared, independently-justified additions (a machine-install probe and an agent_backend governance scope) plus their panel wiring. Every substantive item has a named harm rooted in a real boundary. The one first-principles snag: the agent_sdk package facade re-exports 11 symbols but only 3 have any consumer.

First-Principles-Verdict: CONCERNS

Both features earn their place, but agent_sdk/__init__.py re-exports 8 symbols nothing imports — speculative facade surface behind a one-way door.

What this change ships

Intent: tell an operator why a backend option is dead (install state) and let a fleet bound which harness is selectable. ADDITION (two, both declared in the title).

1. Panel disables an option and names the absent component + install command when the harness isn't installed here — justified (undiagnosable dead-end).
2. Panel shows "check failed" (unknown) but leaves the option enabled — justified (fail-open, avoids bogus reinstall).
3. Panel disables an installed-but-process-cached-absent backend with a restart line — justified (real spawn trap).
4. New owner-only GET /api/acp-backends endpoint — justified; reuses _selectable_acp_backends, no second derivation.
5. agent_backend governance scope, narrowing the registry at gateway start — justified (enterprise boundary, Add an agent_backend governance scope to bound which ACP harness a fleet may select #6622); partial enforcement (no runtime re-derive/session retirement) honestly declared and test-pinned.
6. Footer line "set is fixed at gateway start" — justified (honest substitute for an unattainable freshness signal).
7. agent_sdk facade + drivers/acp.py boundary populated — justified, except the re-export surface below.

Watch

Governance scope binds only at gateway start; runtime apply_ceiling deliberately does not re-narrow (H3). Fully disclosed in code/docs/panel and pinned by test_apply_ceiling_does_not_renarrow_the_registry, so not a defect — but the shipped enforcement is narrower than "a fleet can bound which harness it runs" reads at first glance.

Subtractions

• Trim agent_sdk/__init__.py's re-export/__all__ to the three names with a real consumer (INSTALLED, MISSING, probe_backends — the only ones dashboard/handlers/acp_backend_status.py:94 imports). The other 8 (CACHE_TTL_SECONDS, COMPONENT_KIRO_CLI, COMPONENT_CLAUDE_ACP_ADAPTER, COMPONENT_CLAUDE_CODE_CLI, UNKNOWN, BackendInstallState, clear_probe_cache, probe_backend) have zero consumers — the test imports backend_install directly (test/test_agent_sdk_backend_install.py:39), not the package name.

[FIRST-PRINCIPLES-REVIEWED] 68af3f7

[github-actions]

Opus 4.8 Review — ✅ no blocking findings

Reviewed 68af3f7218072fa27f9daafe5bce597c1998b705 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 68af3f7

Verdict parsed from the review's SHA-scoped output markers for commit 68af3f7218072fa27f9daafe5bce597c1998b705.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 68af3f7218072fa27f9daafe5bce597c1998b705: <one-sentence reason>

[iamwhatever]

• Runtime ceiling refresh is a real enforcement hole, and the PR knowingly ships the wrong primitive for it — fixed in 2ceb8571c594ebc83409026e88e6abd68acde1c9.

Taken as stated, including the part about the primitive. deny_selectable_backend is
gone rather than wrapped: acp_backends now holds the edition-registered baseline
apart from the effective set, and apply_selectable_denials(denied) assigns
baseline - denied whole. That is what makes both directions work — idempotent under
repeated tightening, restoring when a later ceiling is looser — and it leaves no
just-added public function to reshape later. narrow_selectable_backends() now runs
from bootstrap_context and from policy_distribution.apply_ceiling, iterating the
baseline rather than the already-narrowed set. Three tests pin it. See my reply to the
GPT lane on 2ceb8571 for the residual this does NOT cover (live sessions and pooled
providers already running the denied harness).

[iamwhatever]

• The code's enforcement story contradicts the implementation in two authoritative places — fixed in 2ceb8571c594ebc83409026e88e6abd68acde1c9.

Both corrected, and this was the more valuable half of the review. The
agent_backend_governance docstring no longer claims the ceiling is boot-frozen; it
names the two call sites and why a per-decision read is impossible (H3). The
SCOPE_CATALOG comment no longer says the scope is consulted at provider
construction — it now says the opposite explicitly and names H13 plus the test that
asserts create_provider_factory carries no governance call. Your reading of the risk
was right: the old comment was an instruction to a future maintainer to "restore" a
check that would break H13.

[iamwhatever]

• No docs/system-specs update — fixed in 2ceb8571c594ebc83409026e88e6abd68acde1c9.

docs/system-specs/modules/governance.md gains a section for the scope (+151/-5):
policy ids versus the code's spelling, additive-over-floor with the JSON example and
the un-repairable-install reasoning, the #6622 ruling and that #6622 stays open for the
wider enforcement question, the two-call-site recompute with the H3/H4/H13 explanation
of why it cannot be a per-decision read, fail-closed-to-the-floor, and the
both-directions HOST_SESSION_KEY-bound audit. agent_backend is added to the
enforced-scopes paragraph that serves as the file's index, the narrowing to the
_host-sentinel caller list, and the Files/Tests sections updated.

[iamwhatever]

• Governance ceiling not enforced on the runtime refresh path — fixed in 2ceb8571c594ebc83409026e88e6abd68acde1c9.

"Ships partially inert" was accurate and is no longer true for future selection:
narrow_selectable_backends() now runs from policy_distribution.apply_ceiling as
well as bootstrap_context, and the destructive mutator that made a loosening
unrecoverable is gone (apply_selectable_denials assigns baseline - denied). Pinned
by three tests: tighten-at-runtime, loosen-restores, and
narrowing-failure-does-not-roll-back-the-ceiling. One residual remains and is now
disclosed rather than implied — see my reply to the GPT lane on 2ceb8571: sessions
and pooled providers ALREADY running a denied harness are not retired by narrowing.

[iamwhatever]

• Subtraction: drop BACKEND_BY_POLICY_ID (zero consumers) — fixed in 2ceb8571c594ebc83409026e88e6abd68acde1c9.

Correct, and the grep was right: the only reference outside its own definition was a
test assertion, not a consumer — the narrowing only ever translates
backend -> policy_id forward. Both the dict and the assertion are deleted. I had added
it for symmetry with the forward map, which is exactly the kind of surface this lane
exists to catch.

[iamwhatever]

• span=9e5efcca323c — Runtime policy tightening leaves the denied backend usable — fixed in 68af3f7218072fa27f9daafe5bce597c1998b705, by removing the runtime call site.

Taken as stated, and the prescribed direction is the one taken: the
narrow_selectable_backends() call in apply_ceiling is gone. Your reasoning is
right — narrowing the registry bounds future SELECTION and nothing else, so a
tightening applied mid-process would have left sessions and warm-pool providers
running the denied harness while the option vanished from the panel. That reads as
complete enforcement and is not, which is worse than a narrow promise kept: an
operator would take the disappearance as evidence the harness had stopped being used.

Retiring live work needs a session-lifecycle capability that does not exist in this
repo today, so rather than ship a partial runtime enforcement, this scope now promises
only what it can keep: the set is decided at gateway start, and a policy change
binds on the next start. That is not left implicit — it is stated in the dashboard
panel (set_is_fixed_at_gateway_start), in the agent_backend_governance module
docstring, in docs/system-specs/modules/governance.md, and in the PR description.

test_apply_ceiling_does_not_renarrow_the_registry pins the omission in both
directions, with a comment naming what must land alongside the recompute before it is
wired in, so the gap cannot be mistaken for an oversight and quietly closed without
the retirement half. The primitive stays non-destructive
(apply_selectable_denials assigns baseline - denied), so that future change is one
line rather than a redesign.

Ledger note: this is the third blocking finding on this span. Round 2 asked for the
boot-only narrowing to be reverted until a runtime ceiling could recompute it; I did
that, and round 3 asked for the runtime enforcement to be reverted. Those two
prescriptions are mutually exclusive, so rather than apply a fourth point-fix I put the
invariant to the repo owner: a denial is only complete when something retires live work
on the denied harness, and narrowing cannot do that. He chose the explicit
restart-scoped contract, which is what is implemented here.