feat(apps): add an AgentCore Observatory built-in app by warren830 · Pull Request #8463 · kirodotdev/KiroCrew

warren830 · 2026-09-04T13:56:07Z

Problem / Motivation

Amazon Bedrock AgentCore spreads one deployment across several console surfaces.
Answering the two questions an operator actually has — is this runtime healthy,
and is online evaluation actually switched on — means visiting each of them and
correlating by hand. Nothing in the Kiro Crew dashboard reads that estate, so an
AgentCore deployment is invisible from the surface where the rest of the work
happens.

Why it matters

The failure this addresses is not "no dashboard". It is that the answer people
reach for is a count, and a count cannot tell an empty region from a denied
call. Both render as zero. An operator who reads "0 runtimes" and concludes
nothing is deployed, when the truth is that their profile lacks
bedrock-agentcore:ListAgentRuntimes, has been actively misled.

What changed (motivation → approach → change)

Goal: make an AgentCore estate legible from the dashboard without becoming a
second console, and without ever being able to spend money or mutate a resource.

Approach, and what it was chosen over. The obvious shape is a page of summary
cards. I rejected it: a bare Evaluators 32 is exactly the count-versus-denial
ambiguity above, and it answers nothing an operator can act on. The next option
was a hand-written page per resource type — 27 near-identical components, each a
place for the response-key handling to drift.

What is here instead is catalog-driven: backend/catalog.py holds all 27
control-plane types as data — list verb, response key, get verb, identifier
field, and parent linkage — and one query function reads that row to build the
argv. Adding a type is a row, not a branch. The response keys in particular
cannot be inferred and are not guessed: credentialProviders is shared by three
types, items by gateways and gateway targets, and agentRuntimes by both
runtimes and runtime versions. Each was read off the live API.

Only 17 of the 27 types are root-listable. The other 10 require a parent
identifier (and policy-generation-assets requires two), so they are not in the
rail at all; they load from an expanded parent row, which is also the only place
their parent id exists.

Reads are lazy per type, and deliberately not parallel. Listing everything up
front is 27 sequential aws CLI subprocesses. Parallelising them is worse than
slow: concurrent invocations race the same SSO token file.

A section that cannot be read says so. A per-section failure returns HTTP 200
with ok:false so one card degrades instead of the page, and the UI renders an
authorization failure as a denial and a capped list as partial — never as an
account with nothing deployed. This is not theoretical: during live verification
memories hit an SSL UNEXPECTED_EOF_WHILE_READING and the page reported a
failure rather than absence, which is the distinction the app exists to keep.

Configuration is a profile NAME and a region, nothing else. No credential is
read, written or cached — the aws CLI resolves it. Agent sessions cannot reach
these verbs at all: cloud/aws.py's chokepoint already refuses non-allowlisted
verbs when a session key is set, and this change does not widen that allowlist.

One shared-file change: AgentCore is added to the do-not-translate glossary.
Without it the app title tripped the untranslated-script gate in the six
non-Latin locales, which forced a translated product name where all 19 sibling
apps with proper-noun names keep theirs. It is the same mechanism that lets
AWS Control stay English in zh-CN: the DNT term is stripped before the gate
judges the value. Verified to add zero new findings to glossary.test.ts's
baseline — the alternative of adding the whole phrase AgentCore Observatory
would have introduced 21.

Tests

Backend, 140 tests over the app (100% on catalog/query/config, 99% on routes):

test_catalog.py — internal consistency of all 27 rows against the verified
API facts: every type has a list verb or is the documented get-only singleton,
parent params and parent fields are positionally paired, ids are unique.
test_agentcore.py — the argv each listable type builds; error-versus-empty,
truncation, and the unconfigured and malformed-response branches.
test_config.py — profile/region validation and every documented corruption
mode of the stored config.
test_routes.py — all six routes, the enablement gate on each, path-identifier
refusal, and that /catalog performs no AWS call.
test_manifest.py — the manifest invariants the reference apps pin
(default-disabled, no agents, one UI page, declared assets exist on disk).

Frontend, website/src/apps/agentcore-observatory/test/labels.test.ts, 9 tests
pinning the two defects found by driving the real page:

A version row list is keyed uniquely. list-agent-runtime-versions returns the
same agentRuntimeName and the same agentRuntimeArn for every version —
the API reference documents a :version suffix on that ARN that the service
does not send — so keying on either collapsed thirteen rows onto one identity
and expanding one expanded all of them. The test fails if the key stops
including the index.
A version renders as v13, not 13, so it cannot be read as a count.

Manual verification

Driven against a real AWS account in an isolated pod, since no unit test proves a
CLI-backed read works end to end:

The rail's grouped types load lazily; agent-runtimes returned five READY
runtimes, and expanding one loaded its 13 versions and its DEFAULT endpoint.
evaluators and online-evaluation-configs returned real data.
runtimes in an empty region returned ok=true, items=[] and rendered as
"None in this region." rather than as an error.
memories failed with an SSL error and rendered as a failure, not as absence.
Profile and region persist and show as in effect after a reload.

Screenshots / video

Populated state: the grouped rail, one expanded runtime, its 13 versions each
distinguishable by version badge, and one row expanded to raw JSON.

The grey bars are redactions applied before committing: the account id, runtime
ARNs, runtime ids and the runtime name are covered because this is a real
deployment in a public PR. agentRuntimeVersion and status are left readable
on purpose — they are the fields the shot exists to demonstrate.

The same image is committed twice by design: temp-screenshots/ is the PR
evidence copy this repo's convention asks for, and
website/public/app-assets/agentcore-observatory/screenshot-main.png is the
packaged app-store asset the manifest declares and
test_every_discovered_builtin_declares_a_real_screenshot requires.

Related Issues

Fixes #8420

Checklist

At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
Existing tests pass and new tests added for new functionality
Self-review completed; code follows project style guidelines
Documentation updated (if applicable)
No secrets, credentials, or internal references in the diff

Amazon Bedrock AgentCore spreads one deployment across several console surfaces, so answering "is this runtime healthy, and is online evaluation actually on" means visiting each of them. Nothing in the dashboard reads that estate. Adds a default-disabled built-in app that lists AgentCore resources for one AWS profile name and region. The resource surface is data, not code: catalog.py holds all 27 control-plane types with the verb, response key and identifier fields each one needs, so a type is a row rather than a branch. Only the 17 root-listable types appear in the rail; the rest load from a parent row, because their list verb requires a parent id. Reads are lazy per type. Listing all 27 up front is 27 sequential aws CLI subprocesses, and they are deliberately not parallelised because concurrent invocations race the same SSO token file. A section that cannot be read says so. An authorization failure renders as a denial and a truncated page as partial, never as an account with nothing deployed -- a live run surfaced an SSL failure against `memories` and reported it as a failure rather than as absence, which is the distinction the page exists to keep. Configuration is a profile NAME and a region only. No credential is read, written or cached; the aws CLI resolves it, and the cloud chokepoint already refuses these verbs from an agent session. Adds AgentCore to the do-not-translate glossary. Without it the app title tripped the untranslated-script gate in six locales, which forced a translated product name where every sibling app keeps its own -- the same mechanism that lets `AWS Control` stay English in zh-CN.

github-actions · 2026-09-05T02:28:27Z

First Principles Review (Fable 5, fork) — 🔴 BLOCK

Premise-level review of cdbc271c9379ac3df8d12ddab303d1a4021c6159 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All facts verified. The review is complete.

First-Principles-Verdict: BLOCK

The repo's own blocking rule closes the built-in set — a new app ships through the KiroCrewApps registry, and this PR ships one as a built-in anyway.

What this change ships

Intent: let an operator see whether an AgentCore deployment is healthy and evaluated, from the dashboard — an ADDITION.

A new "AgentCore Observatory" app appears in every install's App Store — violates the closed built-in set
Grouped rail of 17 root types, loaded lazily per type — justified
Failures render as denial/error/partial, never as "nothing deployed" — justified (the named harm)
Saved profile name + region, region deliberately undefaulted — justified
Profile-name suggestions from the deploy registry — justified
A /resource/{type}/detail HTTP route and getDetail client — zero consumers
A getConfig client method — zero consumers (/catalog already embeds config)
A profile_default string in 12 locales — zero consumers
"AgentCore" added to the do-not-translate glossary — declared, justified by the i18n gate
Screenshot committed twice (PR evidence + packaged asset) — declared, matches convention

Blockers

New built-in app. AUTOSDE.yaml:496 (no-new-builtin-apps, blocking: true): "The set of built-in apps is CLOSED. A new app ships as an EXTERNAL app published through the KiroCrewApps registry… never as a new built-in in this repo," with the sole exception being "an explicit maintainer decision recorded on the PR." The description never mentions that decision. Subtraction: publish this app through the KiroCrewApps registry, deleting the src/kiro_crew/apps/builtins/agentcore_observatory/ entry, the BUILTIN_NAMES line, builtinRegistry.ts, appManifest.ts, and 12-locale additions — or obtain the recorded override.
Detail endpoint with zero consumers. getDetail (api.ts) is defined and never called; _handle_resource_detail + DetailResponse are exercised only by tests (grepped getDetail: 1 defining site, 0 callers). The UI shows row JSON from the list response. Subtraction: drop the /detail route, getDetail, and DetailResponse.

Subtractions

Drop getConfig in api.ts — 0 callers; /catalog carries the config.
Drop the profile_default key from all 12 locale files — 0 references in any component.

[FIRST-PRINCIPLES-REVIEWED] cdbc271

github-actions · 2026-09-05T02:29:07Z

Design Review (Fable 5, fork) — 🔴 BLOCK (blocking)

Design-level review of cdbc271c9379ac3df8d12ddab303d1a4021c6159 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

I have everything I need. The design is internally well-built (catalog-driven, chokepoint reuse, error-vs-empty distinction), but it ships as a new built-in app, which the repo's governance explicitly closes off — and I verified the external-app path supports the same shape (UI page + backend).

Design-Verdict: BLOCK

A new built-in app is a closed door here: this ships as an external KiroCrewApps registry app, not under apps/builtins/.

Blockers

Wrong distribution vehicle. The patch adds a new app directory + app.json under src/kiro_crew/apps/builtins/agentcore_observatory/ → AGENTS.md and the blocking no-new-builtin-apps rule state "the built-in set is closed, and new apps ship as external apps through the KiroCrewApps registry" → the app rides the wheel into every install, cannot be uninstalled (only disabled), and its 27-row catalog of a preview service's live-observed response keys ("Each was read off the live API") can only be corrected by a Kiro Crew client release instead of a catalog publish — the exact liability the registry exists to avoid, and App Kit's external-app shape (UI page + backend, per docs/app-kit/examples/) supports this app as-is. Fix: republish through the KiroCrewApps registry, or obtain the explicit maintainer override the rule requires.

Watch

A drifted or renamed response key renders as ok=True, items=[] ("None in this region") — payload.get(rt.list_key) silently misses — reproducing the very count-versus-absence ambiguity the PR's stated thesis is to eliminate; consider treating a payload with list-valued keys but no list_key as an error.

[DESIGN-REVIEWED] cdbc271

github-actions · 2026-09-05T02:31:15Z

GPT 5.6 Review (fork) — 🔴 changes requested (blocking)

Reviewed cdbc271c9379ac3df8d12ddab303d1a4021c6159 via the fork AI-review pipeline; updated in place on each push.

2 of 4 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

BLOCKING -- src/kiro_crew/apps/builtins/agentcore_observatory/app.json:2 -- New built-in violates the closed app set
"name": "agentcore-observatory",
New manifest -> built-in discovery -> non-uninstallable app ships inside every wheel.
Anchor: no-new-builtin-apps
Fix: Publish the app through the KiroCrewApps registry.

BLOCKING -- src/kiro_crew/apps/builtins/agentcore_observatory/backend/routes.py:61 -- AWS inventory routes omit owner authorization (origin: validation)
return await handler(request)
Non-owner authenticated session -> enabled wrapper -> gateway AWS profile reads -> cloud inventory disclosed.
Anchor: backend-security-controls
Fix: Add the shared owner check and audit its denial before dispatch.

BLOCKING -- src/kiro_crew/apps/builtins/agentcore_observatory/backend/routes.py:241 -- Query keys inject AWS global options
id_args = {f"--{key}": value for key, value in request.query.items() if key and value}
?endpoint-url=http://127.0.0.1:<port> -> get_resource -> run_aws -> AWS CLI contacts the chosen internal endpoint.
Anchor: residual/security
Fix: Accept only identifier flags explicitly declared for the selected resource type.

BLOCKING -- website/src/apps/agentcore-observatory/AgentcoreObservatoryPage.tsx:73 -- Failures bypass ErrorNotice
<div className="flex gap-2 items-start text-sm">
Query or mutation failure -> Problem -> structured error context and required hand-off decision are lost.
Anchor: errors-use-error-notice
Fix: Use ErrorNotice, explicitly selecting askAgent or documenting the unsaved draft at each call site.

FINDING -- src/kiro_crew/apps/builtins/agentcore_observatory/backend/routes.py:141, src/kiro_crew/apps/builtins/agentcore_observatory/tests/test_manifest.py:99 -- Function-local from ... import load_registry and import ... as pkg violate top-level-imports -> Fix: move both imports to module scope.

[BLOCK-MERGE] cdbc271
[GPT-REVIEWED] cdbc271

Adjudication (Opus 4.8) — is blocking on each finding proportionate?

Both findings are confirmed against the code and anchored to AUTOSDE rules carrying blocking: true, whose authoritative flag outranks weighing.

F1 — New app.json at src/kiro_crew/apps/builtins/agentcore_observatory/app.json (diff line 2323 in patch) adds a new built-in app directory. Anchor no-new-builtin-apps (AUTOSDE.yaml:496–497, blocking: true) flags exactly this. The closed built-in set is a fork invariant restated in AGENTS.md.

F4 — Problem renders query/mutation failures via a hand-written <div className="flex gap-2 items-start text-sm"> (patch line 2368) instead of ErrorNotice. Anchor errors-use-error-notice (website/AUTOSDE.yaml:526–527, blocking: true) covers useQuery/useMutation error surfaces rendered as a bare div. Confirmed match.

[ADJUDICATION] cdbc271c9379ac3df8d12ddab303d1a4021c6159 total=2 uphold=2 downgrade=0
UPHOLD F1 src/kiro_crew/apps/builtins/agentcore_observatory/app.json:2 reason=autosde-blocking-rule
UPHOLD F4 website/src/apps/agentcore-observatory/AgentcoreObservatoryPage.tsx:73 reason=autosde-blocking-rule
[GPT-ADJUDICATED] cdbc271c9379ac3df8d12ddab303d1a4021c6159

github-actions · 2026-09-05T02:32:54Z

Opus 4.8 Review (fork) — 🔴 changes requested (blocking)

Reviewed cdbc271c9379ac3df8d12ddab303d1a4021c6159 via the fork AI-review pipeline; updated in place on each push.

Two merge-blockers: a new app added under apps/builtins/ violates the blocking: true no-new-builtin-apps rule, and the detail route promotes every query key into an arbitrary aws CLI flag (option injection / SSRF).

BLOCKING — src/kiro_crew/apps/builtins/agentcore_observatory/app.json:1 (+ src/kiro_crew/apps/builtins/init.py:4)
+ "agentcore_observatory", adds a brand-new app directory with a new app.json under src/kiro_crew/apps/builtins/.
The no-new-builtin-apps rule is blocking: true and its file-patterns (src/kiro_crew/apps/builtins/**) match these added files → the built-in set is closed; a new app must ship through the external KiroCrewApps registry.
Fix: publish as an external app via the KiroCrewApps registry rather than a new built-in (or land the explicit maintainer /ai-review override, which I cannot and do not read).

BLOCKING — src/kiro_crew/apps/builtins/agentcore_observatory/backend/routes.py:241
id_args = {f"--{key}": value for key, value in request.query.items() if key and value}
An enabled-app request GET .../resource/memories/detail?endpoint-url=http://169.254.169.254/... → every query key becomes --<key>; get_resource's only guard is param.startswith("--") and safe_identifier(value), and safe_identifier permits .:/-, so a URL value passes → [SERVICE, "get-memory", "--endpoint-url", "http://…"] reaches the credentialed aws CLI, redirecting the SigV4-signed request to an attacker/internal host (arbitrary global-flag injection into a surface documented as read-only list/get).
Fix: accept only the identifier flags the catalog declares for the type (per-type allow-list of flag names), as the list route already does via _parent_ids_from_query; do not promote arbitrary query keys to --flags.

FINDING — src/kiro_crew/apps/builtins/agentcore_observatory/backend/agentcore.py:138 — rc, out, err = run_aws(args, ...) never pins --output json (unlike sibling cloud.aws.checked_json), so for any operator whose profile sets output = text/table or AWS_DEFAULT_OUTPUT=text, json.loads(out) raises and every read returns "the aws CLI returned output that is not JSON" on a healthy account → Fix: append ["--output", "json"] to the argv built in _call.

[BLOCK-MERGE] cdbc271
[OPUS-REVIEWED] cdbc271

github-actions · 2026-09-05T02:33:00Z

UX Review (Fable 5, fork) — 🟡 CONCERNS

UX-level review of cdbc271c9379ac3df8d12ddab303d1a4021c6159 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All evidence is in. The change is a new opt-in "AgentCore Observatory" app: a grouped resource rail, lazy per-type lists, a connect form, with honest empty/denied/partial states. Two genuine UX risks survive scrutiny: the hand-rolled error surface renders raw machine codes/CLI stderr (diverging from the product's ErrorNotice/localised-code pattern its sibling aws-control app established), and the packaged App Store screenshot is byte-identical to the redaction-barred PR evidence shot.

UX-Verdict: CONCERNS

Errors reach the user as raw machine codes and CLI stderr, and the App Store showcase screenshot ships with grey redaction bars over its content.

Watch

Hand-rolled Problem shows machine text where every sibling shows a sentence. Typo a region → PUT /config returns code:"invalid_region" → asProblem puts the code in error → user sees "Could not read this" (nothing was read; a save was rejected) + literal invalid_region. Read failures likewise render raw aws stderr, and none offer the product's ErrorNotice "ask the agent" hand-off that aws-control routes every error through. Frequency: first-run setup and every credential failure; impact: friction, off-pattern; persistence: every occurrence. Fix: render via ErrorNotice, map the six backend codes to catalog sentences, and give the connect form its own "save rejected" headline.
screenshot-main.png is the redacted evidence shot, reused "by design" (same blob aee928e56). The PR states grey bars cover the account id, ARNs, ids, and runtime names — so every App Store visitor evaluating the app sees censored rows and cannot tell what it displays. Fix: capture the store asset from a synthetic/demo dataset.

Suggestions

"Connect" only saves two strings — success shows "In effect" with no probe, so a bad profile name surfaces later as stderr; run one cheap list call on save, or relabel to "Use this region".
profile_default ("CLI default") is translated in all 13 locales but never rendered; wire it up as the empty-profile indicator next to "In effect", or drop the key.

[UX-REVIEWED] cdbc271

bolichen97 · 2026-09-08T10:30:52Z

@warren830 Thanks for this. Nothing on main covers it and no other PR duplicates it, so we are not closing it as redundant: origin/main has no AgentCore surface at all (BUILTIN_NAMES holds 14 apps, none of them agentcore_observatory). One note, #8420 was closed by you before this PR was opened, so its closed state is not evidence that the work landed.

What blocks it is the shape, not the idea. no-new-builtin-apps (AUTOSDE.yaml:497) is blocking and closes the built-in set, which is why all four AI lanes block. The rule does allow a recorded maintainer override, and there is precedent: project_scaffolder landed as a new built-in in #8924, after that rule was added in #8009. So please either republish through the KiroCrewApps registry, dropping src/kiro_crew/apps/builtins/agentcore_observatory/, the BUILTIN_NAMES line, the builtinRegistry.ts and appManifest.ts entries and the 13 locale blocks, or ask for the override on this PR.

Three defects need fixing either way:

routes.py:61 _require_enabled checks only is_app_enabled, while aws_control/backend/routes.py:484 also requires is_owner_dashboard_request, so any authenticated non-owner caller can read the operator's AWS inventory.
routes.py:241 promotes arbitrary query keys to aws CLI flags, so ?endpoint-url=... reaches the CLI.
agentcore.py _call never pins --output json, so a profile with output = text reports a healthy account as a JSON parse error.

Also drop the zero-consumer /detail route, getDetail, getConfig and the unreferenced profile_default locale key (website/src/i18n/deadKeys.test.ts enforces that), and route errors through ErrorNotice instead of the hand-rolled Problem component. The branch is 472 commits behind main and shares the new-builtin registration files with #8064, #7877 and #5890, so it needs a rebase.

Posted from the 2026-09-08 open-PR relationship audit (read-only, one auditor per PR); reply here if any of this is wrong.

warren830 requested a review from a team September 4, 2026 13:56

warren830 requested a review from a team as a code owner September 4, 2026 13:56

warren830 requested a review from iamwhatever September 4, 2026 13:56

warren830 force-pushed the feat/agentcore-observatory branch from d13cd30 to cdbc271 Compare September 5, 2026 02:15

github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 5, 2026

github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 5, 2026

bolichen97 mentioned this pull request Sep 8, 2026

feat(apps): add chat-status-tags builtin — SDLC + health tagging for dashboard chats #7877

Open

Navigation Menu

Uh oh!

feat(apps): add an AgentCore Observatory built-in app - #8463

feat(apps): add an AgentCore Observatory built-in app#8463
warren830 wants to merge 1 commit into
kirodotdev:mainfrom
warren830:feat/agentcore-observatory

warren830 commented Sep 4, 2026

Uh oh!

github-actions Bot commented Sep 5, 2026

Uh oh!

github-actions Bot commented Sep 5, 2026

Uh oh!

github-actions Bot commented Sep 5, 2026

Uh oh!

github-actions Bot commented Sep 5, 2026

Uh oh!

github-actions Bot commented Sep 5, 2026

Uh oh!

bolichen97 commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

warren830 commented Sep 4, 2026

Problem / Motivation

Why it matters

What changed (motivation → approach → change)

Tests

Manual verification

Screenshots / video

Related Issues

Checklist

Uh oh!

github-actions Bot commented Sep 5, 2026

First Principles Review (Fable 5, fork) — 🔴 BLOCK

What this change ships

Blockers

Subtractions

Uh oh!

github-actions Bot commented Sep 5, 2026

Design Review (Fable 5, fork) — 🔴 BLOCK (blocking)

Blockers

Watch

Uh oh!

github-actions Bot commented Sep 5, 2026

GPT 5.6 Review (fork) — 🔴 changes requested (blocking)

Uh oh!

github-actions Bot commented Sep 5, 2026

Opus 4.8 Review (fork) — 🔴 changes requested (blocking)

Uh oh!

github-actions Bot commented Sep 5, 2026

UX Review (Fable 5, fork) — 🟡 CONCERNS

Watch

Suggestions

Uh oh!

bolichen97 commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants