diff --git a/.agents/skills/verify-agent-server/references/feature-map/F02-auth-and-sessions.md b/.agents/skills/verify-agent-server/references/feature-map/F02-auth-and-sessions.md index d64a6bd19e1..7df70e738b0 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F02-auth-and-sessions.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F02-auth-and-sessions.md @@ -105,7 +105,9 @@ Preconditions: network namespace (`unshare -rn`, so nothing outside it can connect, and port 8000 is free in every replay), it runs this checkout's server the way an operator would (`python -m openhands.agent_server`, no `--host` unless - given, default port 8000) with its own `HOME` and config, waits until + given, default port 8000) with its own `HOME` and config and + `LITELLM_LOCAL_MODEL_COST_MAP=True` (the namespace has no network, and a + failed cost-map fetch at `import litellm` can crash startup, #6749), waits until `/proc/net/tcp` shows the listening socket, stops the server, prints its log and `listening on ADDRESS:8000`, and fails if nothing listened. Needs `unshare` (util-linux) with unprivileged user namespaces, or root. @@ -143,7 +145,7 @@ Preconditions: cd "$dir" || exit 1 printf '{"preload_tools": false, "enable_vscode": false}\n' > config.json HOME="$dir/home" OH_PERSISTENCE_DIR="$dir/home/.openhands" OPENHANDS_AGENT_SERVER_CONFIG_PATH="$dir/config.json" \ - COLUMNS=400 PYTHONUNBUFFERED=1 "$QA_F02_PYTHON" -m openhands.agent_server "$@" > server.log 2>&1 & + LITELLM_LOCAL_MODEL_COST_MAP=True COLUMNS=400 PYTHONUNBUFFERED=1 "$QA_F02_PYTHON" -m openhands.agent_server "$@" > server.log 2>&1 & pid=$! addr= for _ in $(seq 240); do @@ -546,6 +548,13 @@ Preconditions: commands run as the server's user and can read `/proc//environ`. - `restart --env` values persist for later restarts of the same run; this family leaves `OH_SESSION_API_KEYS_1` and the CORS variables on its run. +- `bind.sh` sets `LITELLM_LOCAL_MODEL_COST_MAP=True` because the private + namespace has no network: without it LiteLLM's remote cost-map fetch fails + at import and startup crashes most of the time with an `ImportError` + (`LoggerContext` from a partially initialized + `litellm.rust_bridge.catalog`) or a `_DeadlockError` (#6749, also hit by + `F24.offline-safe`), so `F02.default-bind` would fail for a reason + unrelated to the bind address. - With keys configured and no `--host`, the server binds `0.0.0.0`; without keys, `127.0.0.1` (`F02.default-bind`). The launcher always passes `--host 127.0.0.1`, so only `F02.default-bind` sees the default. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F03-deferred-init.md b/.agents/skills/verify-agent-server/references/feature-map/F03-deferred-init.md index 7b78b4aa28e..e9560e416f4 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F03-deferred-init.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F03-deferred-init.md @@ -173,8 +173,8 @@ Preconditions: rewritten by the 5xx handler, so assert on the status and on `exception`, which carries `server is in deferred-init state 'dormant'; call POST /api/init first`. The workspace file route is mounted on its own `/api` - router (cookie auth), which carries the same gate since #5700, so it no - longer reaches the missing conversation service. + router (cookie auth), which carries the same gate, so it never reaches + the missing conversation service. - **Dormant sockets (`F03.dormant-gates-sockets`).** Every WebSocket is refused at the handshake while dormant. ```sh F03_ZERO=00000000-0000-0000-0000-0000000000f3 @@ -557,11 +557,9 @@ Preconditions: client as an HTTP 403 handshake rejection (`ws listen --expect-reject 403`), not as a 1013 close frame. An unknown socket path is also a 403, so pair the dormant check with the same socket opening after init. -- The gate precedes every auth check since #5700 (`F03.gate-before-auth`), so +- The gate precedes every auth check (`F03.gate-before-auth`) and covers the + workspace static-file routes, the sockets, `/v1` and `/app-backends`, so while dormant a 503 says nothing about the key; prove a key only after init. - Before #5700 the workspace static-file routes, the sockets, `/v1` and - `/app-backends` were outside the gate (#5661, #5700 closed them); a baseline - checkout older than those PRs answers those recipes differently. - The bootstrap `X-Init-API-Key` is the dormant `secret_key`, which defaults to the first session key when `OH_SECRET_KEY` is unset and is **open to anyone** when neither is set (`F03.init-open-without-secret`). `control-agent-server diff --git a/.agents/skills/verify-agent-server/references/feature-map/F04-conversation-lifecycle.md b/.agents/skills/verify-agent-server/references/feature-map/F04-conversation-lifecycle.md index 264732934e6..e8cdf8f831d 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F04-conversation-lifecycle.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F04-conversation-lifecycle.md @@ -51,7 +51,7 @@ Routes: `GET /api/conversations/search`, `GET /api/conversations/count`, - `F04.patch-errors`: an empty or 201-character title, a bad tag key and a malformed id are 422; no key is 401; the stored values are untouched. - `F04.patch-whitespace-title`: a whitespace-only title is rejected instead of erasing the stored title. - `F04.patch-unknown-id`: PATCH on an unknown id answers the 404 the OpenAPI document declares, not 200 `{"success": false}`. -- `F04.delete`: DELETE stops a running conversation (`PauseEvent`, `InterruptEvent`), removes its state, keeps the workspace, removes a worktree conversation's worktree but keeps its branch, orphans children, and later reads are 404 (socket close 4004); a malformed id is 422 and no key is 401. +- `F04.delete`: DELETE closes the conversation's open sockets with 4004, stops a running conversation, removes its state, keeps the workspace, removes a worktree conversation's worktree but keeps its branch, orphans children, and later reads are 404 (socket close 4004); a malformed id is 422 and no key is 401. - `F04.delete-unknown-id`: DELETE on an unknown or already deleted id answers the 404 the OpenAPI document declares, not 400. - `F04.restart-persist`: after a restart the catalog, statuses, titles, tags, final responses and event replay are unchanged, and deleted conversations stay deleted. - `F04.read-no-hydrate`: after a restart, GET, the batch get, search and count read a cold conversation without loading it (no `owner_lease.json`), while `agent_final_response` loads it and claims the lease. @@ -85,9 +85,9 @@ Routes: `GET /api/conversations/search`, `GET /api/conversations/count`, `DELETE /api/conversations/{conversation_id}`. - WebSocket (owned by the events family, used here as a second view): `/sockets/events/{conversation_id}` replays the initial message and run, - pushes `ConversationStateUpdateEvent` key `tags` on PATCH, `PauseEvent` and - `InterruptEvent` when a running conversation is deleted, and closes with 4004 - for a deleted id. + pushes `ConversationStateUpdateEvent` key `tags` on PATCH, closes an open + socket with 4004 when its conversation is deleted, and closes with 4004 for + a deleted id. - SDK: `RemoteConversation(agent=..., workspace=RemoteWorkspace(...))` posts the create (and re-attaches with `conversation_id`), reads state with `GET /api/conversations/{id}`, `set_title()` sends the PATCH, and `close()` @@ -656,8 +656,7 @@ EOF N4=$(control-agent-server api GET /api/conversations/count --field .) control-agent-server ws start "/sockets/events/$DEL" --name qa-f04-delete --duration 60 control-agent-server api DELETE "/api/conversations/$DEL" --expect 200 --check success eq true --timeout 30 --save F04.delete/delete - control-agent-server ws read qa-f04-delete --kinds PauseEvent --expect-min 1 --wait 15 - control-agent-server ws stop qa-f04-delete --kinds InterruptEvent --expect-min 1 --wait 15 --save F04.delete/live + control-agent-server ws stop qa-f04-delete --kinds PauseEvent,InterruptEvent --expect-none --expect-close 4004 --save F04.delete/live control-agent-server api GET "/api/conversations/$DEL" --expect 404 --save F04.delete/get-after control-agent-server api GET "/api/conversations/$DEL/agent_final_response" --expect 404 control-agent-server api GET /api/conversations --query ids="$DEL" --check 0 eq null @@ -673,8 +672,10 @@ EOF if git -C "$REPO" worktree list | grep -q "worktrees/$WT/"; then false; fi test -n "$(git -C "$REPO" branch --list "openhands/$WT")" ``` - The running conversation is paused and interrupted, then gone from every - read view and from disk; the workspace file and the child (with a dangling + The open socket is closed with 4004 `Conversation deleted` before the + DELETE answers and before the running conversation is paused and + interrupted, so it receives no `PauseEvent` or `InterruptEvent`; the + conversation is then gone from every read view and from disk; the workspace file and the child (with a dangling `parent_conversation_id`) remain. The worktree conversation's worktree is removed (directory and `git worktree` entry) while its branch stays. A repeated delete and an unknown id are `F04.delete-unknown-id`. @@ -896,9 +897,10 @@ EOF - `parent_conversation_id` needs the same `working_dir` as the parent. DELETE does not cascade: children keep a dangling `parent_conversation_id`, and a worktree and its `openhands/` branch stay behind. -- An events socket that was already open during a DELETE receives the pause - frames and then a 4004 close before the DELETE answers; new connections - get 4004 too (`F06.ws-delete-close`). +- An events socket that was already open during a DELETE gets a 4004 close + before the DELETE answers, and before the run is paused, so it sees no + pause or interrupt frames (`F04.delete`); new connections get 4004 too + (`F06.ws-delete-close`). - Saved LLM profiles cannot be named in `agent_settings`; start from one with `agent_profile_id` (an Agent Profile whose `llm_profile_ref` names it), as in `F04.create-agent-profile`. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F05-run-pause-interrupt.md b/.agents/skills/verify-agent-server/references/feature-map/F05-run-pause-interrupt.md index e9687460de5..d5bfb30afe4 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F05-run-pause-interrupt.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F05-run-pause-interrupt.md @@ -46,13 +46,14 @@ Routes: `POST /api/conversations/{conversation_id}/run`, - `F05.interrupt-tool-call`: interrupt while a tool call runs returns at once: `paused`, an `InterruptEvent`, an `AgentErrorEvent` for the unfinished call (same `tool_call_id`) and no observation; the resumed run sends that error to the model as the tool result and finishes. - `F05.run-capacity`: with `max_concurrent_runs` 1, `POST /run` on a second conversation returns 429 while the first holds the slot, leaves it `idle`, and succeeds once the first is interrupted. - `F05.stuck-detected`: with `stuck_detection` on (the default), a run whose model repeats the same tool call ends `stuck` after four identical action/observation pairs (four model calls), with no `ConversationErrorEvent`; GET, `status=stuck` search and count and the events socket say `stuck`. -- `F05.stuck-rerun`: `POST /run` on a `stuck` conversation with no new input returns 200, goes `running` and straight back to `stuck` without a model call. The SDK reports the same behavior as a known failure ([#6648](https://github.com/OpenHands/software-agent-sdk/issues/6648), `S01.stuck-recover`): if that is fixed, this bullet changes with it. +- `F05.stuck-rerun`: `POST /run` on a `stuck` conversation with no new input returns 200 and resumes it with a fresh stuck-detection window: the model is called again, and a model that keeps looping ends `stuck` again after four more identical action/observation pairs (as the SDK does, `S01.stuck-recover`). - `F05.stuck-new-input`: a new user message moves a `stuck` conversation to `idle`, and `POST /run` runs it again to `finished`. - `F05.max-iterations-stop`: a conversation created with `max_iterations: 6` and `stuck_detection: false` whose model never finishes makes exactly six model calls (past the stuck threshold) and ends `error` with exactly one `ConversationErrorEvent` `code: MaxIterationsReached` naming the limit, on REST and the events socket, never `stuck`. - `F05.max-iterations-resume`: `POST /run` after `MaxIterationsReached` is accepted and gets a fresh budget of six model calls (a second `MaxIterationsReached`), and once the model calls `finish` the next run ends `finished`. - `F05.restart-running-graceful`: a conversation running at a graceful restart comes back `paused`, with a `PauseEvent` and an `InterruptEvent`. - `F05.restart-keeps-status`: `finished`, `paused` and `error` statuses survive a restart unchanged. - `F05.restart-hard-running`: a conversation running when the server is SIGKILLed comes back `error` on the next start, and `/run` resumes it. +- `F05.restart-hard-tool-call`: a conversation SIGKILLed while a tool call runs comes back `error` with one `AgentErrorEvent` for that `tool_call_id` that says where it came from: `recovery_reason` `server_restart` and `classification` `{"kind": "internal", "retryable": false}`, on search, get-one and the events socket replay; an interrupted call's `AgentErrorEvent` has no `recovery_reason`. - `F05.pause-running-crash`: a pause accepted during a model call survives a SIGKILL: the conversation comes back `paused`, as its persisted `PauseEvent` and status update say, not `error`. ## How to get to it (agent POV) @@ -65,7 +66,7 @@ Routes: `POST /api/conversations/{conversation_id}/run`, `{"detail": "credential_binding_activation_required", "retryable": true}` when a required credential binding is missing (credential family); 400 while the server shuts down. `/run` also resumes `paused` and `error` - conversations (a `stuck` one only after new input) and approves actions + conversations (a `stuck` one with a fresh stuck-detection window) and approves actions waiting for confirmation. - REST: `POST /api/conversations/{conversation_id}/pause` and `POST /api/conversations/{conversation_id}/interrupt` (no body), 200 @@ -584,13 +585,16 @@ control-agent-server api GET /api/conversations/count --query status=idle --chec control-agent-server ws read f05-l-rerun --contains '"key": "execution_status", "value": "running"' control-agent-server ws stop f05-l-rerun --contains '"key": "execution_status", "value": "stuck"' --save F05.stuck-rerun/frames control-agent-server api GET "/api/conversations/$L" --check execution_status eq stuck - control-agent-server api GET "/api/conversations/$L/events/count" --query kind=$ACTION_KIND --check . eq 4 - control-agent-server sink read --name qa-f05-loop --expect-min 4 --expect-max 4 + control-agent-server api GET "/api/conversations/$L/events/count" --query kind=$ACTION_KIND --check . eq 8 + control-agent-server api GET "/api/conversations/$L/events/count" --query kind=$OBSERVATION_KIND --check . eq 8 + control-agent-server sink read --name qa-f05-loop --expect-min 8 --expect-max 8 ``` - `/run` is accepted and the run ends at once (its closing `full_state` - frame): the socket shows `running` then `stuck`, and there is no new action - and no fifth model request. The stuck detector looks only at events since - the last user message, so `/run` alone re-detects the same loop. + `/run` is accepted and the run resumes: the stuck detector counts only the + events after the resume (as it does after a new user message), so the + four earlier pairs do not re-trigger it. The stub keeps looping, so after + four more model calls, actions and observations (eight of each in all) + the run ends `stuck` again (its closing `full_state` frame): the socket + shows `running` then `stuck`. - **New input un-sticks it (`F05.stuck-new-input`).** The provider now answers with a `finish` call. ```sh @@ -600,7 +604,7 @@ control-agent-server api GET /api/conversations/count --query status=idle --chec control-agent-server api POST "/api/conversations/$L/run" --expect 200 --check success eq true control-agent-server conversation wait "$L" --until finished --timeout 60 control-agent-server api GET "/api/conversations/$L/agent_final_response" --check response eq QA_F05_UNSTUCK --save F05.stuck-new-input/final - control-agent-server sink read --name qa-f05-loop --expect-min 5 --expect-max 5 + control-agent-server sink read --name qa-f05-loop --expect-min 9 --expect-max 9 ``` The message resets `stuck` to `idle`, and the next run makes one model call and ends `finished` with the stub's `finish` message. @@ -697,6 +701,45 @@ control-agent-server api GET /api/conversations/count --query status=idle --chec over the dead owner's lease, marks `K` `error` (a persisted `execution_status` update), nothing is `running`, and `/run` resumes `K` (interrupted again to free the slot). +- **Crash during a tool call (`F05.restart-hard-tool-call`).** A new + conversation `X` on its own stub, whose model asks for `terminal` + `sleep 120`, is SIGKILLed while that command runs (the slot is free: + `K` was interrupted). `T`'s interrupted call is the contrast. + + Requires: `F05.interrupt-tool-call` + Needs: `tmux` + ```sh + control-agent-server api GET "/api/conversations/$T/events/search" --query kind=$AGENT_ERROR_KIND \ + --check items len-eq 1 --check items.0.recovery_reason missing --save F05.restart-hard-tool-call/interrupt-error + CRASH=$(control-agent-server fixture llm-stub --name qa-f05-crash --step 'tool:terminal:{"command":"sleep 120"}' --print-path) + CRASH_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key\", \"base_url\": \"$CRASH/v1\", \"num_retries\": 0}, \"tools\": [{\"name\": \"terminal\"}]}" + X=$(control-agent-server conversation start --body-json "{\"agent\": $CRASH_AGENT}" --no-autotitle --print-id) + control-agent-server conversation send "$X" --text 'hi' --no-run + control-agent-server api POST "/api/conversations/$X/run" --expect 200 --check success eq true + control-agent-server api GET "/api/conversations/$X/events/count" --query kind=$ACTION_KIND --check . eq 1 --until-ok 30 + XCALL=$(control-agent-server api GET "/api/conversations/$X/events/search" --query kind=$ACTION_KIND --field items.0.tool_call_id) + control-agent-server api GET "/api/conversations/$X" --check execution_status eq running + control-agent-server restart --hard + control-agent-server api GET "/api/conversations/$X" --check execution_status eq error --save F05.restart-hard-tool-call/get + control-agent-server api GET "/api/conversations/$X/events/search" --query kind=$AGENT_ERROR_KIND \ + --check items len-eq 1 --check items.0.tool_call_id eq "$XCALL" --check items.0.recovery_reason eq server_restart \ + --check items.0.classification.kind eq internal --check items.0.classification.retryable eq false \ + --check items.0.error contains 'A restart occurred while this tool was in progress' --save F05.restart-hard-tool-call/agent-error + XERR=$(control-agent-server api GET "/api/conversations/$X/events/search" --query kind=$AGENT_ERROR_KIND --field items.0.id) + control-agent-server api GET "/api/conversations/$X/events/$XERR" --check recovery_reason eq server_restart --save F05.restart-hard-tool-call/get-one + control-agent-server api GET "/api/conversations/$X/events/count" --query kind=$OBSERVATION_KIND --check . eq 0 + control-agent-server ws listen "/sockets/events/$X" --query resend_mode=all --duration 5 \ + --until-kind AgentErrorEvent --until recovery_reason=server_restart --save F05.restart-hard-tool-call/replay + control-agent-server doctor + ``` + Before the crash `X` is `running` with one `ActionEvent` and no result. + After it, `X` is `error` and carries exactly one `AgentErrorEvent` + answering `XCALL` ("A restart occurred while this tool was in + progress..."), marked `recovery_reason` `server_restart` and classified + `internal`, not retryable, the same on search, get-one and the socket + replay, and no observation. `T`'s `AgentErrorEvent` from an interrupt has + no `recovery_reason` (search omits null fields), so a consumer can tell a + crash-synthesized outcome from an interrupted one. - **Crash after a mid-call pause (`F05.pause-running-crash`).** `K` runs into the hanging stub again, is paused while the model call is in flight, and the server is SIGKILLed. The last bullet, so nothing depends on @@ -765,7 +808,8 @@ control-agent-server api GET /api/conversations/count --query status=idle --chec - Graceful shutdown pauses and cancels running conversations (`PauseEvent` then `InterruptEvent`); SIGKILL leaves `running` on disk, and the next start turns it into `error` and appends an `AgentErrorEvent` for an unfinished - tool call, if any. A lease held by a dead process on the same host is taken + tool call, if any, marked `recovery_reason: "server_restart"` + (`F05.restart-hard-tool-call`). A lease held by a dead process on the same host is taken over immediately; a lease from another host must expire first (`OH_LEASE_TTL_SECONDS`, default 45), and until then `/run` answers 404. - `F05.run-error-responsive` only means something with the default console @@ -778,11 +822,12 @@ control-agent-server api GET /api/conversations/count --query status=idle --chec after every pause and interrupt. Counting them (`--expect-min`) is how the recipes prove a run ended when no status changes (`F05.run-finished-noop`, `F05.pause-tool-step`). -- `stuck` is not an error: no `ConversationErrorEvent`, and `/run` on a - `stuck` conversation re-runs the detector over the events since the last - user message, so it returns to `stuck` at once without a model call. Only - new input (or a conversation created with `stuck_detection: false`) lets - the loop continue (`F05.stuck-rerun`, `F05.stuck-new-input`). +- `stuck` is not an error: no `ConversationErrorEvent`. `/run` on a + `stuck` conversation resumes it with a fresh detection window (only events + after the resume count), so a model that keeps repeating itself makes four + more calls and ends `stuck` again; new input also resets the window + (`F05.stuck-rerun`, `F05.stuck-new-input`). A conversation created with + `stuck_detection: false` never ends `stuck`. - `max_iterations` counts agent steps per run, not per conversation: every `/run` starts from zero (`F05.max-iterations-resume`). A step that ends with `finish` on the last allowed iteration stays `finished`. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F06-conversation-events.md b/.agents/skills/verify-agent-server/references/feature-map/F06-conversation-events.md index 1d6cf702e0b..66e63c49c3f 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F06-conversation-events.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F06-conversation-events.md @@ -52,10 +52,12 @@ Routes: `GET /api/conversations/{conversation_id}/events/search`, - `F06.ws-replay-since`: `resend_mode=since` with `after_timestamp` replays exactly the events at or after that timestamp; without `after_timestamp` it replays nothing. - `F06.ws-bad-frame`: an invalid inbound frame returns a `ServerErrorEvent` (`ValidationError`, `AssertionError`, `JSONDecodeError` for text that is not JSON), appends nothing and leaves the socket open; a redundant auth frame is ignored. - `F06.ws-send`: a user message frame sent on the socket is persisted and runs the agent to `finished`, streaming the run on the same socket. -- `F06.restart-persist`: event ids, order and count survive a server restart, the socket replays the same log, and new messages append after it. +- `F06.restart-persist`: a socket open during a server restart is closed with 1012; event ids, order and count survive the restart, the socket replays the same log, and new messages append after it. - `F06.send-run-limit`: with every run slot taken, `POST .../events` with `run: true` returns 429 `Message saved...` and the message is persisted; a socket send gets a `ConversationRunLimitExceeded` `ServerErrorEvent`. - `F06.send-while-running`: `POST .../events` with `run: true` on a conversation that is already running returns 200 (not the 409 of `POST /run`), persists the message, keeps the conversation `running` and starts no second run. +- `F06.send-during-tool-call`: `POST .../events` to a conversation whose agent is inside a long tool call saves the message and answers at once, before the tool call ends. + ## How to get to it (agent POV) - REST: `POST /api/conversations/{conversation_id}/events` with a @@ -593,7 +595,11 @@ Preconditions: ```sh BEFORE=$(control-agent-server api GET "/api/conversations/$CID/events/search" --field items | jq -c '[.[].id]') COUNT=$(control-agent-server api GET "/api/conversations/$CID/events/count" --field .) + control-agent-server ws start "/sockets/events/$CID" --name f06-restart --duration 120 + control-agent-server ws read f06-restart --kinds ConversationStateUpdateEvent --contains '"key": "full_state"' \ + --expect-open --wait 10 control-agent-server restart + control-agent-server ws stop f06-restart --expect-close 1012 --save F06.restart-persist/socket control-agent-server api GET "/api/conversations/$CID/events/count" --check . eq "$COUNT" --save F06.restart-persist/count AFTER=$(control-agent-server api GET "/api/conversations/$CID/events/search" --field items | jq -c '[.[].id]') test "$BEFORE" = "$AFTER" @@ -606,8 +612,12 @@ Preconditions: --check items.1.llm_message.content.0.text eq QA_F06_AFTER_RESTART control-agent-server api GET "/api/conversations/$CID/events/count" --check . gt "$COUNT" ``` - The first read after the restart rehydrates the conversation; ids, order - and count are unchanged, the socket replays the same log, and the new + A socket open across the restart is closed with 1012 (service restart, a + code clients reconnect after) and an empty reason: uvicorn closes every + upgraded socket that way when it starts shutting down, before the app's + lifespan shutdown would send its own 1012 `Server shutting down`. The + first read after the restart + rehydrates the conversation; ids, order and count are unchanged, the socket replays the same log, and the new message lands after it. - **Run capacity full (`F06.send-run-limit`).** It restarts the run with one run slot and holds that slot with a scripted provider that hangs (a real @@ -698,6 +708,38 @@ Preconditions: stored or replayed: clients use the final durable message after reconnect, and keep transient tokens out of their event-log cache. +- **A message during a tool call (`F06.send-during-tool-call`), known bug #6729.** + Last bullet. A scripted model asks for `terminal` `sleep 20` on every + call (a real model cannot be held inside a tool call on demand); once its + `ActionEvent` is logged and no observation is, a message is appended with + `run: false`. The run slot is free: `HOLD` was interrupted and the + streamed conversation finished. + + Needs: `tmux` + ```sh + TOOL_STUB=$(control-agent-server fixture llm-stub --name qa-f06-tool --step 'tool:terminal:{"command":"sleep 20"}' --print-path) + TOOL_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key\", \"base_url\": \"$TOOL_STUB/v1\", \"num_retries\": 0}, \"tools\": [{\"name\": \"terminal\"}]}" + TC=$(control-agent-server conversation start --body-json "{\"agent\": $TOOL_AGENT}" --no-autotitle --print-id) + control-agent-server api POST "/api/conversations/$TC/events" \ + --json '{"content":[{"type":"text","text":"QA_F06_TOOL"}],"run":true}' --expect 200 + control-agent-server api GET "/api/conversations/$TC/events/count" --query kind=ActionEvent --check . eq 1 --until-ok 30 + control-agent-server api GET "/api/conversations/$TC/events/count" --query kind=ObservationEvent --check . eq 0 + control-agent-server api POST "/api/conversations/$TC/events" \ + --json '{"content":[{"type":"text","text":"QA_F06_DURING_TOOL"}],"run":false}' --expect 200 --expect-max-ms 3000 --timeout 60 \ + --save F06.send-during-tool-call/post # bug: status eq 200 && elapsed_ms ge 15000 + control-agent-server api GET "/api/conversations/$TC/events/count" --query body=QA_F06_DURING_TOOL --check . eq 1 + control-agent-server api GET "/api/conversations/$TC/events/count" --query kind=ObservationEvent --check . eq 0 + control-agent-server api POST "/api/conversations/$TC/interrupt" --expect 200 + control-agent-server conversation wait "$TC" --until paused --timeout 60 + ``` + Expected: the POST answers within 3 s and the message is in the log + while the command still runs (no observation yet); the agent sees it at + its next step. Today the POST waits for the state lock, which the run + loop holds for the whole tool call, so it answers 200 only when + `sleep 20` ends, about 20 s later (#6729; the same lock makes `/pause` + wait, F05). The message is then saved. A user's correction to a busy + agent waits as long as the running command. + ## Gotchas - `kind` filters take either the bare class name events carry in `kind` @@ -749,7 +791,8 @@ Preconditions: fatal. - Deleting a conversation closes its open sockets with 4004 (`F06.ws-delete-close`), and a new connection gets 4004 too; shutting the - server down closes them with 1012. A sandbox pause leaves them attached + server down closes them with 1012 and an empty reason, sent by uvicorn + before the app's own shutdown close (`F06.restart-persist`). A sandbox pause leaves them attached (`F12.sandbox-pause-websocket`). A peer that stopped reading delays the DELETE by at most 5 s. - `StreamingDeltaEvent` frames reach this socket only when the agent's LLM diff --git a/.agents/skills/verify-agent-server/references/feature-map/F07-session-socket.md b/.agents/skills/verify-agent-server/references/feature-map/F07-session-socket.md index 23fe053f42c..240b77aa4b1 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F07-session-socket.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F07-session-socket.md @@ -24,7 +24,9 @@ Routes: `WS /sockets/session/{conversation_id}` ## Sub-features - `F07.sync-first`: the first server frame is `{"type": "sync"}`; `from_seq` echoes `after_seq` (absent when live-only) and `through_seq` is the last on-disk `seq` (absent for an empty log). +- `F07.sync-first-during-run`: while a run holds the conversation's state lock, `sync` is still the first frame; an `error` reply to an invalid frame sent right after connecting comes after it. - `F07.snapshot-transient`: every connection gets one `transient` frame with a `full_state` `ConversationStateUpdateEvent` and no `seq`, after `sync` (and after the replay when one ran); it is never persisted. +- `F07.snapshot-compact-stats`: the `full_state` snapshot carries each LLM's aggregate stats (`accumulated_cost`, `accumulated_token_usage`) but not the per-call `costs`, `response_latencies` and `token_usages` lists, so it stays small however long the conversation runs. - `F07.full-replay`: `after_seq=-1` (or any value below it) replays every persisted event as `durable` frames with `seq` 0..`through_seq`, in order, once each, with the same ids REST returns. - `F07.seq-on-disk`: a durable frame's `seq` is the event's on-disk index, the `NNNNN` in `events/event-NNNNN-.json`. - `F07.resume-cursor`: `after_seq=k` replays exactly the events with `seq > k`; `after_seq=through_seq` replays nothing. @@ -446,6 +448,32 @@ Preconditions: ``` The replay carries the durable reply (its id is `ITEM`) and no progress frame of any kind. +- **Compact snapshot stats (`F07.snapshot-compact-stats`), known bug #6741.** + Reconnect live-only to the streamed conversation, which has made one + model call, and read its snapshot's stats for the agent's LLM (usage id + `default`). + + Requires: `F07.progress-frames` + ```sh + control-agent-server ws listen "/sockets/session/$SID" --count 2 --save "F07.snapshot-compact-stats/live-$SID" + control-agent-server state cat "evidence/F07.snapshot-compact-stats/live-$SID.json" \ + --check frames.2.frame.event.key eq full_state \ + --check frames.2.frame.event.value.stats.usage_to_metrics.default.accumulated_token_usage.prompt_tokens gt 0 + control-agent-server state cat "evidence/F07.snapshot-compact-stats/live-$SID.json" \ + --check frames.2.frame.event.value.stats.usage_to_metrics.default.accumulated_cost exists \ + --check frames.2.frame.event.value.stats.usage_to_metrics.default.response_latencies missing \ + --check frames.2.frame.event.value.stats.usage_to_metrics.default.token_usages missing \ + --check frames.2.frame.event.value.stats.usage_to_metrics.default.costs missing # bug: checks.0.ok eq true && checks.1.actual contains latency + ``` + Expected: the snapshot keeps the aggregates (prompt tokens above 0, an + `accumulated_cost`) and drops the per-call lists, as the incremental + `stats` updates do (`use_snapshot`). Today the `full_state` snapshot is + dumped without that context, so it carries one `costs`, + `response_latencies` and `token_usages` entry per model call (about + 2.6 KB each); past roughly 1,550 calls the snapshot exceeds the 4 MiB + frame cap and every connection closes with 1013 `frame_too_large` + before the client sees anything (#6741). The legacy events socket's + snapshot (F06) is built the same way. - **Interrupted stream (`F07.item-aborted`).** Start a streamed deepseek-flash answer that takes tens of seconds (counting to 3000), wait until its `delta` frames flow on a background capture, then interrupt the @@ -649,8 +677,31 @@ Preconditions: arrived by the time the capture is stopped. The legacy events socket is closed the same way (`F06.ws-delete-close`). +- **Sync first while a run holds the lock (`F07.sync-first-during-run`), known bug #6745.** + Start a run whose agent sits in a 25 s `sleep`, connect while the tool + call is in flight, and send an `assistant` message at once. + + Needs: `tmux` + ```sh + RUNC=$(control-agent-server conversation start --no-autotitle --prompt 'Run exactly this terminal command and nothing else: sleep 25 && echo QA_F07_LOCK' --print-id) + control-agent-server conversation wait "$RUNC" --until running --timeout 60 + control-agent-server api GET "/api/conversations/$RUNC/events/search" --query kind=ActionEvent --check items len-ge 1 --until-ok 60 --quiet + control-agent-server ws listen "/sockets/session/$RUNC" --send '{"role":"assistant","content":[{"type":"text","text":"QA_F07_LOCK_ROLE"}]}' \ + --until-kind sync --duration 90 --expect-kind error --save "F07.sync-first-during-run/order-$RUNC" + control-agent-server state cat "evidence/F07.sync-first-during-run/order-$RUNC.json" --check frames.2.kind eq sync # bug: checks.0.actual eq error + control-agent-server conversation wait "$RUNC" --until finished,idle --timeout 120 + ``` + Expected: `sync` arrives first (when the step ends and the lock frees), + then the `error` frame (`AssertionError`, `Only user messages are allowed + to be sent to the agent.`). Today the socket reads client frames from the + moment it connects, before subscribing (which waits for the lock), so the + `error` frame is sent at once and `sync` only when the step ends, about + 25 s later (#6745). A valid user message sent during the step is saved + either way. + ## Gotchas +- `F07.reject-non-user`, `F07.error-frame` and `F07.auth-frame-ignored` read the `error` frames at fixed indexes after `sync` and the snapshot. While #6745 lasts, an `error` can overtake `sync` when the server is slow to subscribe (seen under load, for `F07.auth-frame-ignored` as `frames.3.frame.type` `error`), and the bullet then fails on its first index check; rerun it alone with `--only`. `F07.sync-first-during-run` reproduces that order deterministically. - Neither the SDK nor the TypeScript client uses this socket yet, so a regression here breaks no shipped consumer's tests; only these recipes and `tests/agent_server/test_session_socket.py` notice. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F09-model-switching-plugins.md b/.agents/skills/verify-agent-server/references/feature-map/F09-model-switching-plugins.md index f6016d5512d..072192c19dc 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F09-model-switching-plugins.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F09-model-switching-plugins.md @@ -3,8 +3,8 @@ What a consumer changes on a conversation that already exists, without recreating it: swap the agent's LLM for a caller-supplied config (inline key, a cipher token from an `encrypted` read, or a saved provider connection), -switch it to a saved LLM profile, change the model an ACP conversation's -subprocess runs, and load a plugin from the conversation's registered +switch it to a saved LLM profile, change the model and the reasoning +effort an ACP conversation's subprocess runs, and load a plugin from the conversation's registered marketplaces into the live agent (its skills, commands and hooks). Every switch answers `{"success": true}`, is written to the conversation's `base_state.json`, shows up in `GET /api/conversations/{id}` and as a @@ -19,11 +19,12 @@ Needs: `llm`, `tmux`, `node`, `network` Routes: `POST /api/conversations/{conversation_id}/switch_profile`, `POST /api/conversations/{conversation_id}/switch_llm`, `POST /api/conversations/{conversation_id}/load_plugin`, -`POST /api/conversations/{conversation_id}/switch_acp_model` +`POST /api/conversations/{conversation_id}/switch_acp_model`, +`POST /api/conversations/{conversation_id}/switch_acp_effort` ## Sub-features -- `F09.auth-required`: all four routes answer 401 without a valid session key and change nothing. +- `F09.auth-required`: all five routes answer 401 without a valid session key and change nothing. - `F09.switch-llm`: `switch_llm` with a fresh `usage_id` answers 200 `{"success": true}`; `GET` and `base_state.json` show the new model and `usage_id`, the condenser LLM that mirrored the old one follows, and the key is never echoed or stored in plaintext. - `F09.switch-llm-ws`: a switch pushes a `ConversationStateUpdateEvent` with key `agent` and the new model on the events socket, with the key masked. - `F09.switch-llm-errors`: an unknown conversation is 404, a body without `llm` or without `llm.model` is 422, and a dangling `provider_connection_id` is 422; the installed LLM is unchanged. @@ -43,8 +44,17 @@ Routes: `POST /api/conversations/{conversation_id}/switch_profile`, - `F09.acp-model-live`: on an ACP conversation with a live session, `switch_acp_model` switches the subprocess's model in place (same ACP session) and `current_model_id`, `acp_model` and `base_state.json` follow; a blank model is 400, a model the provider does not offer is refused and changes nothing, and the next turn runs on the switched model in the same session. - `F09.acp-model-live-unknown`: a live switch to a model id the provider does not offer (not in `available_models`) is a client error (400, as for a blank model), not a 500. - `F09.acp-model-live-timeout`: a live switch whose `session/set_model` round-trip outlasts `acp_prompt_timeout` is 504 with `detail` `Internal Server Error` and leaves the model unchanged. +- `F09.acp-effort-native`: `switch_acp_effort` on a regular (non-ACP) conversation is 400 `switch_acp_effort is only supported for ACP conversations.`; a missing `effort` is 422 and an unknown conversation 404. +- `F09.acp-effort-deferred`: on an ACP conversation that has not run yet, `switch_acp_effort` answers 200 without starting the subprocess and persists the level as `agent.acp_effort`, `agent_state.acp_current_effort` and `current_effort` (no level list exists before a session, so any non-blank level is stored); a blank level is 400. +- `F09.acp-model-id-bare`: on a live ACP session `current_model_id` is the bare model id: a legacy `/` `acp_model` (`sonnet/high`) still starts the session, `current_model_id` is `sonnet`, and an explicit `acp_effort` wins over the suffix in `current_effort`. +- `F09.acp-effort-live`: on a live ACP session, `switch_acp_effort` to a level in `available_efforts` answers 200 and switches the session in place (same `acp_session_id`); `current_effort`, `agent.acp_effort`, `agent_state.acp_current_effort` and `base_state.json` follow, and the next turn finishes in the same session. +- `F09.acp-effort-unoffered`: on a live session, a level the current model does not offer is 400 naming the offered levels, a blank level is 400, and `current_effort` and `acp_effort` are unchanged. +- `F09.acp-effort-model-switch`: after `switch_acp_model` to another model (`haiku`) and back (`sonnet`), the pinned `acp_effort` is kept and applied on each model that offers it (`current_effort` follows, `available_efforts` lists that model's levels), in the same session. +- `F09.acp-effort-live-timeout`: a live `switch_acp_effort` whose round-trip outlasts `acp_prompt_timeout` is 504 with `detail` `Internal Server Error` and leaves the effort unchanged. +- `F09.acp-effort-codex`: on a Codex (`codex`) ACP conversation signed in with a ChatGPT subscription, `available_efforts` lists the levels the current model offers and `switch_acp_effort` switches among them live (blocked: needs a ChatGPT login). - `F09.ts-client`: the TypeScript client's `ConversationClient.switchLLM`, `switchProfile` and `switchAcpModel` and `ConversationManager.switchProfile` reach the routes: each switch resolves and reads back through `getConversation`, an unknown profile rejects with 404, `switchAcpModel` on a regular conversation with 400, a call without a key with 401, and `switchAcpModel` on an ACP conversation that has not run resolves and stores the model. - `F09.ts-client-acp-docs`: the TypeScript client's `switchAcpModel` docstrings describe what the server does before an ACP conversation's first run (200, the model is stored for the first session), not a 409. +- `F09.ts-client-acp-effort`: the TypeScript client's `ConversationClient.switchAcpEffort` stores the level on an ACP conversation that has not run (`getConversation` shows `acp_effort` and `current_effort`), and rejects with 400 on a regular conversation and with 401 without a key. - `F09.load-plugin`: `load_plugin` with `@` from a local marketplace registered on the agent's `agent_context` answers 200 and merges the plugin's skill, its command and its hooks into the live agent, visible in `GET`, `base_state.json` and on the socket. - `F09.load-plugin-hook-runs`: after a runtime load, the next user turn runs the plugin's `UserPromptSubmit` hook (a `HookExecutionEvent` with its output) and finishes. - `F09.load-plugin-errors`: no registered marketplaces and no agent context are 400, a registered marketplace whose source is missing is 400, an unknown marketplace or plugin is 404, empty and malformed refs are 400, a missing `plugin_ref` is 422 and an unknown conversation 404; a bare plugin name resolves across the registered marketplaces. @@ -58,10 +68,12 @@ Routes: `POST /api/conversations/{conversation_id}/switch_profile`, `POST /api/conversations/{id}/switch_llm` `{"llm": {...LLM fields...}}`, `POST /api/conversations/{id}/switch_profile` `{"profile_name": "..."}`, `POST /api/conversations/{id}/switch_acp_model` `{"model": "..."}`, + `POST /api/conversations/{id}/switch_acp_effort` `{"effort": "..."}`, `POST /api/conversations/{id}/load_plugin` `{"plugin_ref": "[@]"}`. Each answers `{"success": true}`; every recipe below drives them. - Second views: `GET /api/conversations/{id}` (`agent.llm`, `agent.condenser.llm`, `stats.usage_to_metrics.`, `current_model_id`, `agent.acp_model`, + `current_effort`, `available_efforts`, `agent.acp_effort`, `agent_state`, `hook_config`, and `agent.agent_context.skills` with `include_skills=true`), the conversation's `base_state.json` (`state cat`), `WS /sockets/events/{conversation_id}` (`ConversationStateUpdateEvent` keys @@ -72,22 +84,26 @@ Routes: `POST /api/conversations/{conversation_id}/switch_profile`, `POST /api/conversations` and `POST /api/conversations/{id}/events` (conversations and runs). - SDK: `LocalConversation.switch_llm(llm)`, `switch_profile(name)`, - `switch_acp_model(model)` and `load_plugin(ref)` are what the routes call; + `switch_acp_model(model)`, `switch_acp_effort(effort)` and `load_plugin(ref)` + are what the routes call (the SDK side of the effort switch is + `S21.effort-switch` in the SDK map); over HTTP `RemoteConversation.load_plugin(ref)` posts to `load_plugin` (`F09.load-plugin-sdk` drives it). - `RemoteConversation` has no `switch_llm`, `switch_profile` or - `switch_acp_model` method; Python consumers post to the routes directly. + `RemoteConversation` has no `switch_llm`, `switch_profile`, + `switch_acp_model` or `switch_acp_effort` method; Python consumers post to the routes directly. - TypeScript client: `ConversationClient.switchProfile(id, name)`, - `switchLLM(id, llm)`, `switchAcpModel(id, model)`; - `RemoteConversation.switchProfile`, `switchLlm`, `switchAcpModel`; + `switchLLM(id, llm)`, `switchAcpModel(id, model)`, + `switchAcpEffort(id, effort)`; + `RemoteConversation.switchProfile`, `switchLlm`, `switchAcpModel`, + `switchAcpEffort`; `ConversationManager.switchProfile`. There is no TypeScript `loadPlugin`. `F09.ts-client` drives the `ConversationClient` methods and `ConversationManager.switchProfile` (the `RemoteConversation` methods post the same bodies to the same routes). - Agent Canvas (context only): the per-conversation model picker calls `switch_profile` (or `switch_llm` with a profile config that may reference a - provider connection), and `switch_acp_model` for ACP conversations, picking - from `available_models`. + provider connection), and `switch_acp_model` / `switch_acp_effort` for ACP + conversations, picking from `available_models` and `available_efforts`. ## Driving it with control-agent-server @@ -120,6 +136,7 @@ Preconditions: control-agent-server api POST "/api/conversations/$CID/switch_llm" --auth none --json '{"llm": {"model": "openai/qa-f09-noauth", "usage_id": "qa-f09-noauth"}}' --expect 401 --check detail eq Unauthorized --save F09.auth-required/switch-llm control-agent-server api POST "/api/conversations/$CID/switch_profile" --auth none --json '{"profile_name": "deepseek-pro"}' --expect 401 control-agent-server api POST "/api/conversations/$CID/switch_acp_model" --auth bad --json '{"model": "opus"}' --expect 401 + control-agent-server api POST "/api/conversations/$CID/switch_acp_effort" --auth none --json '{"effort": "high"}' --expect 401 control-agent-server api POST "/api/conversations/$CID/load_plugin" --auth none --json '{"plugin_ref": "qa-f09-mkt-plugin@qa-f09-mkt"}' --expect 401 control-agent-server api GET "/api/conversations/$CID" --check agent.llm.model eq deepseek/deepseek-flash --check agent.llm.usage_id eq default --quiet ``` @@ -485,8 +502,8 @@ Preconditions: ``` After the first turn the conversation reports `supports_runtime_model_switch: true` and the adapter's - `available_models` (`default`, `opus[1m]`, `sonnet`, `sonnet[1m]`, - `haiku`), on `sonnet`. The switch to `haiku` is 200 and `current_model_id`, + `available_models` (`default`, `opus`, `fable`, `sonnet`, `haiku`), on + `sonnet`. The switch to `haiku` is 200 and `current_model_id`, `agent.acp_model`, `agent_state.acp_current_model_id` and `base_state.json` follow, with the same `acp_session_id` (switched in place, not a new session). A blank model is 400 `model must be a @@ -497,26 +514,27 @@ Preconditions: `haiku`. Every built-in provider declares runtime switching, so the route's other 400 (`does not support runtime model switching`) is not reachable with them. -- **Unknown model on a live session (`F09.acp-model-live-unknown`), known bug.** +- **Unknown model on a live session (`F09.acp-model-live-unknown`).** `LID` from the bullet above, whose adapter lists its models. Requires: `F09.acp-model-live` ```sh control-agent-server api GET "/api/conversations/$LID" --quiet --check supports_runtime_model_switch eq true \ --check available_models contains '"model_id": "sonnet"' --check available_models not-contains qa-f09-nope - control-agent-server api POST "/api/conversations/$LID/switch_acp_model" --json '{"model": "qa-f09-nope"}' --expect 400,422 --save F09.acp-model-live-unknown/switch # bug: status eq 500 && response.body.exception contains 'Internal error' - ``` - Expected: a client error, as `ACPAgent.set_acp_model` documents ("the ACP - server rejects the model-switch call (e.g. ... an invalid model id)" is a - `ValueError`, which the route turns into 400) and as a blank model already - is. Today it is 500 `Internal Server Error` with `exception` `Internal - error`: claude-agent-acp answers `set_config_option(model)` with an id it - does not offer with JSON-RPC -32603, and `set_acp_model` re-raises -32603 - as a server-side failure instead of a `ValueError`, although the session's - `available_models` already shows the id is not offered. A consumer that - mistypes a model sees a retriable-looking 500 for a request that can - never succeed. Nothing changes either way (`F09.acp-model-live` checks - that the model stays `haiku`). + control-agent-server api POST "/api/conversations/$LID/switch_acp_model" --json '{"model": "qa-f09-nope"}' --expect 400 \ + --check detail contains 'Invalid value for config option model: qa-f09-nope' --save F09.acp-model-live-unknown/switch + control-agent-server api GET "/api/conversations/$LID" --quiet --check current_model_id eq haiku --check agent.acp_model eq haiku + ``` + A client error, as `ACPAgent.set_acp_model` documents ("the ACP server + rejects the model-switch call (e.g. ... an invalid model id)" is a + `ValueError`, which the route turns into 400) and as a blank model is: + claude-agent-acp answers `set_config_option(model)` with JSON-RPC -32602 + `Invalid params`, and the `detail` carries the adapter's message (`ACP + server rejected set_config_option(model)(model='qa-f09-nope'): Invalid + params: Invalid value for config option model: qa-f09-nope`). The model + stays `haiku`. `set_acp_model` still re-raises an adapter's -32603 + (`Internal error`) as a server-side failure (500), so an adapter that + answered an unknown id with -32603 would surface as a 500 here. - **Live switch that times out (`F09.acp-model-live-timeout`).** The same translated provider, on a conversation created with `acp_prompt_timeout: 0.001`: the server's own timer is what is under test. @@ -534,6 +552,137 @@ Preconditions: answers 504 with `detail` rewritten to `Internal Server Error` (the server's 5xx handler; `exception` is `504: `), and the conversation keeps `sonnet` everywhere: a timed-out switch changes nothing locally. +- **ACP effort on a regular conversation (`F09.acp-effort-native`).** `CID` + runs an OpenHands `Agent`. + ```sh + control-agent-server api POST "/api/conversations/$CID/switch_acp_effort" --json '{"effort": "high"}' \ + --expect 400 --check detail eq 'switch_acp_effort is only supported for ACP conversations.' --save F09.acp-effort-native/native + control-agent-server api POST "/api/conversations/$CID/switch_acp_effort" --json '{}' --expect 422 --check detail.0.loc.1 eq effort + control-agent-server api POST /api/conversations/00000000-0000-4000-8000-000000000000/switch_acp_effort --json '{"effort": "high"}' --expect 404 + control-agent-server api GET "/api/conversations/$CID" --quiet --check agent.kind eq Agent --check current_effort eq null --check available_efforts len-eq 0 + ``` + 400 with the exact message, 422 without `effort`, 404 for the unknown id; + the conversation reports no effort. +- **ACP effort before the first run (`F09.acp-effort-deferred`).** A Claude + Code ACP conversation with `acp_model: sonnet`, no `acp_effort` and no + prompt, so no subprocess starts. + ```sh + EAID=$(control-agent-server conversation start --no-autotitle --body-json '{"agent_settings": {"agent_kind": "acp", "acp_server": "claude-code", "acp_model": "sonnet"}}' --print-id) + control-agent-server api GET "/api/conversations/$EAID" --quiet --check agent.kind eq ACPAgent --check agent.acp_effort eq null \ + --check current_effort eq null --check available_efforts len-eq 0 --check execution_status eq idle + control-agent-server api POST "/api/conversations/$EAID/switch_acp_effort" --json '{"effort": "high"}' --expect 200 --check success eq true --save F09.acp-effort-deferred/switch + control-agent-server api GET "/api/conversations/$EAID" --quiet --check agent.acp_effort eq high --check current_effort eq high \ + --check agent_state.acp_current_effort eq high --check agent_state.acp_session_id missing --check current_model_id eq sonnet \ + --check execution_status eq idle --save F09.acp-effort-deferred/get + control-agent-server state cat "server/workspace/conversations/${EAID//-/}/base_state.json" --max-chars 200 \ + --check agent.acp_effort eq high --check agent_state.acp_current_effort eq high + control-agent-server api POST "/api/conversations/$EAID/switch_acp_effort" --json '{"effort": " "}' --expect 400 --check detail eq 'effort must be a non-empty string' + control-agent-server api GET "/api/conversations/$EAID" --quiet --check agent.acp_effort eq high --check current_effort eq high + ``` + The switch is 200 although no ACP session exists; `acp_effort`, + `agent_state.acp_current_effort` and `current_effort` say `high`, so the + first session starts on it. Before a session there is no level list to + check against, so only a blank level is refused (400), and it changes + nothing. +- **Bare model id with a legacy effort suffix (`F09.acp-model-id-bare`).** + The translated Claude Code provider of `F09.acp-model-live`, on a + conversation created with the legacy `acp_model: sonnet/high` and an + explicit `acp_effort: low`; one turn starts the session. + ```sh + EID=$(control-agent-server conversation start --no-autotitle --body-json '{"agent_settings": {"agent_kind": "acp", "acp_server": "claude-code", "acp_model": "sonnet/high", "acp_effort": "low"}}' \ + --secret ANTHROPIC_API_KEY="$DEEPSEEK_API_KEY" --secret ANTHROPIC_BASE_URL=https://api.deepseek.com/anthropic --secret IS_SANDBOX=1 \ + --prompt 'Reply with the single word: ok' --wait --until finished --timeout 300 --print-id) + ESESS=$(control-agent-server api GET "/api/conversations/$EID" --check execution_status eq finished --check current_model_id eq sonnet \ + --check agent.acp_model eq sonnet/high --check agent.acp_effort eq low --check current_effort eq low \ + --check available_efforts contains '"value": "high"' --check available_models contains '"model_id": "sonnet"' \ + --save F09.acp-model-id-bare/get --field agent_state.acp_session_id) + test -n "$ESESS" + ``` + The session starts on `sonnet` and `current_model_id` is the bare + `sonnet`, an id that appears in `available_models`; the `/high` suffix is + still accepted as input, but `acp_effort: low` wins, so `current_effort` + is `low`. `available_efforts` lists what the adapter offers for `sonnet` + (`default`, `low`, `medium`, `high`, `xhigh`, `max`). +- **Live effort switch (`F09.acp-effort-live`).** Switch `EID`'s live + session to `high`, then run a turn. + + Requires: `F09.acp-model-id-bare` + ```sh + control-agent-server api POST "/api/conversations/$EID/switch_acp_effort" --json '{"effort": "high"}' --expect 200 --check success eq true --save F09.acp-effort-live/switch + control-agent-server api GET "/api/conversations/$EID" --quiet --check current_effort eq high --check agent.acp_effort eq high \ + --check agent_state.acp_current_effort eq high --check agent_state.acp_session_id eq "$ESESS" --check current_model_id eq sonnet --save F09.acp-effort-live/get + control-agent-server state cat "server/workspace/conversations/${EID//-/}/base_state.json" --max-chars 200 --check agent.acp_effort eq high --check agent_state.acp_current_effort eq high + control-agent-server conversation send "$EID" --text 'Reply with the single word: ok' --wait --until finished,error --timeout 300 + control-agent-server api GET "/api/conversations/$EID" --until-ok 30 --quiet --check execution_status eq finished --check agent_state.acp_session_id eq "$ESESS" \ + --check current_effort eq high --save F09.acp-effort-live/second-turn + ``` + The switch is 200, `current_effort`, `agent.acp_effort`, + `agent_state.acp_current_effort` and `base_state.json` say `high`, the + `acp_session_id` is unchanged (switched in place), and the next turn + finishes in the same session, still on `high`. +- **Unoffered effort on a live session (`F09.acp-effort-unoffered`).** + + Requires: `F09.acp-effort-live` + ```sh + control-agent-server api GET "/api/conversations/$EID" --quiet --check available_efforts not-contains ultra --check current_effort eq high + control-agent-server api POST "/api/conversations/$EID/switch_acp_effort" --json '{"effort": "ultra"}' --expect 400 \ + --check detail eq "Reasoning effort 'ultra' is not offered for the current model; choose one of: default, low, medium, high, xhigh, max." --save F09.acp-effort-unoffered/ultra + control-agent-server api POST "/api/conversations/$EID/switch_acp_effort" --json '{"effort": " "}' --expect 400 --check detail eq 'effort must be a non-empty string' + control-agent-server api GET "/api/conversations/$EID" --quiet --check current_effort eq high --check agent.acp_effort eq high --check agent_state.acp_current_effort eq high + ``` + A level the adapter does not offer for `sonnet` is 400 and the detail + lists the offered levels; a blank level is 400; the effort stays `high`. +- **Effort across a model switch (`F09.acp-effort-model-switch`).** Switch + `EID` (effort pinned to `high`) to `haiku` and back to `sonnet`. + + Requires: `F09.acp-effort-live` + ```sh + control-agent-server api POST "/api/conversations/$EID/switch_acp_model" --json '{"model": "haiku"}' --expect 200 + control-agent-server api GET "/api/conversations/$EID" --quiet --check current_model_id eq haiku --check current_effort eq high \ + --check available_efforts contains '"value": "high"' --check agent.acp_effort eq high --check agent_state.acp_current_effort eq high \ + --check agent_state.acp_session_id eq "$ESESS" --save F09.acp-effort-model-switch/haiku + control-agent-server api POST "/api/conversations/$EID/switch_acp_model" --json '{"model": "sonnet"}' --expect 200 + control-agent-server api GET "/api/conversations/$EID" --quiet --check current_model_id eq sonnet --check current_effort eq high \ + --check available_efforts contains '"value": "max"' --check agent_state.acp_session_id eq "$ESESS" --save F09.acp-effort-model-switch/back + ``` + On `haiku` the adapter lists the same levels (`default` to `max`), and + `current_effort` and the pinned `acp_effort` stay `high`. Back on `sonnet` + `current_effort` is `high` again, in the same session. The server re-applies + a pinned effort after a model switch when the new model offers it; on a + model that offers no levels `current_effort` would read null and + `switch_acp_effort` 400 `The current model offers no reasoning effort + setting.`, but every model the pinned claude-agent-acp lists offers + levels, so that path is not reachable with the built-in providers. +- **Live effort switch that times out (`F09.acp-effort-live-timeout`).** + `TOID` from `F09.acp-model-live-timeout` (`acp_prompt_timeout: 0.001`, + live session on `sonnet`, no pinned effort). + + Requires: `F09.acp-model-live-timeout` + ```sh + control-agent-server api GET "/api/conversations/$TOID" --quiet --check agent_state.acp_session_id exists --check available_efforts contains '"value": "high"' \ + --check current_effort eq default --check agent.acp_effort eq null + control-agent-server api POST "/api/conversations/$TOID/switch_acp_effort" --json '{"effort": "high"}' --expect 504 --check detail eq 'Internal Server Error' --save F09.acp-effort-live-timeout/switch + control-agent-server api GET "/api/conversations/$TOID" --quiet --check current_effort eq default --check agent.acp_effort eq null --check agent_state.acp_current_effort eq default + ``` + The session runs at the adapter's own `default` effort. The adapter cannot + answer within 1 ms, so the switch is 504 (`detail` rewritten to `Internal + Server Error`) and nothing changes locally: `current_effort` stays + `default` and `acp_effort` unset. +- **Codex effort levels (`F09.acp-effort-codex`), blocked: needs a ChatGPT subscription login for codex-acp (`CODEX_AUTH_JSON`).** + With the login + passed as the `CODEX_AUTH_JSON` conversation secret, a `codex` conversation + runs one turn; `available_efforts` should list the levels the current + model offers (a model such as `gpt-6.1-sol` also offers `ultra`), and + `switch_acp_effort` to one of them should switch the live session as in + `F09.acp-effort-live`. DeepSeek cannot stand in here: codex-acp signs in + only with an OpenAI key or a ChatGPT login. + ```sh + CXID=$(control-agent-server conversation start --no-autotitle --body-json '{"agent_settings": {"agent_kind": "acp", "acp_server": "codex"}}' \ + --secret CODEX_AUTH_JSON="$QA_CODEX_AUTH_JSON" --prompt 'Reply with the single word: ok' --wait --until finished --timeout 300 --print-id) + control-agent-server api GET "/api/conversations/$CXID" --quiet --check available_efforts contains '"value": "high"' + control-agent-server api POST "/api/conversations/$CXID/switch_acp_effort" --json '{"effort": "high"}' --expect 200 --save F09.acp-effort-codex/switch + control-agent-server api GET "/api/conversations/$CXID" --quiet --check current_effort eq high --check agent.acp_effort eq high + ``` - **TypeScript client (`F09.ts-client`).** `ConversationClient` and `ConversationManager` from the built client in `clients/typescript/dist` (built here when missing, which needs `npm` and the network), on a fresh @@ -602,6 +751,41 @@ Preconditions: `F09.ts-client` observe: before the first run the server answers 200 and stores the model for the first session, and a non-ACP conversation gets 400. Neither promises a 409. +- **TypeScript effort switch (`F09.ts-client-acp-effort`).** + `ConversationClient.switchAcpEffort` from the built client, on a fresh ACP + conversation `AID5` that never runs and a fresh placeholder conversation + `TID2`. + ```sh + test -f clients/typescript/dist/clients.js || (cd clients/typescript && npm ci && npm run build) + TID2=$(control-agent-server conversation start --placeholder-agent --tools none --no-autotitle --print-id) + AID5=$(control-agent-server conversation start --no-autotitle --body-json '{"agent_settings": {"agent_kind": "acp", "acp_server": "claude-code", "acp_model": "sonnet"}}' --print-id) + F="$AGENT_SERVER_VERIFY_RUN/fixtures" + cat > "$F/qa-f09-ts-effort.mjs" <<'JS' + import assert from 'node:assert/strict'; + import { resolve } from 'node:path'; + import { pathToFileURL } from 'node:url'; + + const { ConversationClient } = await import(pathToFileURL(resolve('clients/typescript/dist/clients.js')).href); + const [tid, acp] = process.argv.slice(2); + const host = process.env.AGENT_SERVER_URL; + const client = new ConversationClient({ host, apiKey: process.env.SESSION_API_KEY }); + await client.switchAcpEffort(acp, 'high'); + const info = await client.getConversation(acp); + assert.equal(info.agent.acp_effort, 'high'); + assert.equal(info.current_effort, 'high'); + await assert.rejects(client.switchAcpEffort(tid, 'high'), (e) => e.status === 400); + await assert.rejects(new ConversationClient({ host }).switchAcpEffort(acp, 'low'), (e) => e.status === 401); + assert.equal((await client.getConversation(acp)).agent.acp_effort, 'high'); + console.log('TS_EFFORT_OK'); + JS + control-agent-server exec --expect-output TS_EFFORT_OK --save F09.ts-client-acp-effort/program -- node "$F/qa-f09-ts-effort.mjs" "$TID2" "$AID5" + control-agent-server state cat "server/workspace/conversations/${AID5//-/}/base_state.json" --max-chars 200 --check agent.acp_effort eq high + ``` + `switchAcpEffort` on the ACP conversation that has not run resolves and + `getConversation` shows `acp_effort` and `current_effort` `high` (also in + `base_state.json`); on the regular conversation it rejects with an + `HttpError` 400 and without a key with 401, and the stored level is + unchanged. - **Load a plugin (`F09.load-plugin`).** Start `PID` from the saved settings with the `qa-f09-mkt` marketplace registered on its `agent_context` (not auto-loaded), then load the plugin while capturing the socket. @@ -751,6 +935,7 @@ Preconditions: ## Gotchas +- Agent Canvas does not switch or show the ACP effort yet; its side is [OpenHands/OpenHands#19436](https://github.com/OpenHands/OpenHands/pull/19436), and it also needs an Agent Server release after 1.61.0. - Every switch answers `{"success": true}` and nothing else; read the result back with `GET /api/conversations/{id}` (which serves the autosaved `base_state.json`, so on an idle conversation it shows a switch @@ -805,6 +990,20 @@ Preconditions: Error`; read `exception`. The TypeScript client's `switchAcpModel` docstrings document this 200-and-defer behaviour (`F09.acp-model-deferred`, `F09.ts-client`, `F09.ts-client-acp-docs`). +- `switch_acp_effort` before the first run stores any non-blank level, since + no level list exists yet; the first session applies it only if the model + offers it (otherwise the server logs a warning and the session keeps the + adapter's default), and `current_effort` then reports what the session + runs. On a live session the level must be in `available_efforts`, which + are per model (with claude-agent-acp every listed model offers `default` + to `max`); `acp_effort` stays pinned across a model switch and is + re-applied when the new model offers it. With nothing pinned the session runs + at the adapter's own `default` level (`F09.acp-effort-deferred`, + `F09.acp-effort-model-switch`, `F09.acp-effort-live-timeout`). +- `current_model_id` is the bare model id (`sonnet`), also for a legacy + `acp_model` such as `sonnet/high`; a cold read before the first session + falls back to the stored `acp_model`, suffix included + (`F09.acp-model-id-bare`). - A live ACP session needs the provider's CLI and an account for it. For `claude-code`, DeepSeek's Anthropic-compatible API works as the account (secrets `ANTHROPIC_BASE_URL=https://api.deepseek.com/anthropic` and diff --git a/.agents/skills/verify-agent-server/references/feature-map/F10-fork-navigate-condense-ask.md b/.agents/skills/verify-agent-server/references/feature-map/F10-fork-navigate-condense-ask.md index 00f341a20c1..4014dc4e62c 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F10-fork-navigate-condense-ask.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F10-fork-navigate-condense-ask.md @@ -275,7 +275,7 @@ SIDE_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key N_A_MSG=$(control-agent-server api GET "/api/conversations/$A/events/count" --query kind=$MSG --field .) AF=$(control-agent-server api POST "/api/conversations/$A/fork" --json '{"title": "QA F10 memory fork"}' --expect 201 --field id) control-agent-server conversation send "$AF" --text 'What is the project codename I told you? Answer with the codename only.' --wait --until finished,error --timeout 180 - control-agent-server conversation events "$AF" --kinds ConversationErrorEvent --contains LLMAuthenticationError --expect-count 0 --save F10.fork-run-memory/auth-errors # bug: matching ge 1 + control-agent-server conversation events "$AF" --kinds ConversationErrorEvent --contains Authentication --expect-count 0 --save F10.fork-run-memory/auth-errors # bug: matching ge 1 control-agent-server api GET "/api/conversations/$AF" --quiet --check execution_status eq finished --save F10.fork-run-memory/status control-agent-server api GET "/api/conversations/$AF/agent_final_response" --check response matches '(?i)pelican' --save F10.fork-run-memory/final control-agent-server api GET "/api/conversations/$A/events/count" --query kind=$MSG --check . eq "$N_A_MSG" @@ -286,9 +286,13 @@ SIDE_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key fact only the source was told, not wording; `agent_final_response` reads a reply given as a message or through `finish`), and `A` gains no message from the fork's run. Today the fork's run ends in `error` within seconds - with a `ConversationErrorEvent` `LLMAuthenticationError` ("Your LLM API key - appears to be invalid or has expired."), because the fork lost the model - key (`F10.fork-keeps-credentials`); the marked check fails on that event. + with a `ConversationErrorEvent` for the provider's authentication failure, + because the fork lost the model key (`F10.fork-keeps-credentials`); the + marked check fails on that event. DeepSeek answers the keyless request + with `Authentication Fails (governor)` (`"type":"authentication_error"`), + which the SDK classifies as `LLMBadRequestError` rather than + `LLMAuthenticationError` (the classification is `F20.validate-bad-key-type`), + so the check matches `Authentication`, which both forms carry. Any other failure (a provider outage, a wrong answer) fails a later, unmarked check and reports `fail`, not `xfail`. Any fork of a conversation whose model needs a key cannot run. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F12-runtime-and-credentials.md b/.agents/skills/verify-agent-server/references/feature-map/F12-runtime-and-credentials.md index 6bab04080e9..b204e8d50d6 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F12-runtime-and-credentials.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F12-runtime-and-credentials.md @@ -367,11 +367,12 @@ control-agent-server api GET "/api/conversations/$R" --check execution_status eq `QA_F12_AFTER_PAUSE` message that the fresh sockets get (the events socket keeps its one initial `ConversationStateUpdateEvent`, the session socket its `sync` frame and initial state; still the same 20 s later). The pause - closes the conversation's pub/sub without closing the socket subscribers - (`_WebSocketSubscriber` and the session socket's `_SessionSubscriber` - inherit the no-op `Subscriber.close()`), leaving the clients attached to a - discarded `EventService`. Idle eviction avoids this by never evicting a - conversation with an external subscriber; the pause has no such guard. + exits each `EventService` without calling its `close_connections` (which + delete and shutdown use, with 4004 and 1012), so the clients stay attached + to a discarded service; the code skips the close on purpose, so that a + reconnect does not load the conversation again before the sandbox freezes. + Idle eviction avoids this by never evicting a conversation with an + external subscriber; the pause has no such guard. Both captures are saved before the assertions, so the evidence holds both. - **Unsupported secret names (`F12.binding-unsupported-secret`).** Only `CODEX_AUTH_JSON` is a file credential. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F15-git.md b/.agents/skills/verify-agent-server/references/feature-map/F15-git.md index 2cea527e7fd..72025c4452a 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F15-git.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F15-git.md @@ -14,7 +14,10 @@ operation except repository search also exists under `/api/conversations/{runtime_conversation_id}/git/...`, where `path` must resolve inside the conversation's workspace. Repository search is not about local directories: it lists the repositories a GitHub token stored in the -server's secrets can access (`missing_token` when there is none). +server's secrets can access (`missing_token` when there is none), and +pull-request status reads one GitHub pull request's state and the combined CI +result of its head commit with the same token (409 `missing_provider_token` +when there is none), for status badges. Source: `openhands-agent-server/openhands/agent_server/git_router.py`, `openhands-agent-server/openhands/agent_server/git_provider_service.py`, `openhands-agent-server/openhands/agent_server/runtime_router.py`, `openhands-sdk/openhands/sdk/git/`, `openhands-sdk/openhands/sdk/workspace/remote/remote_workspace_mixin.py`, `clients/typescript/src/workspace/remote-workspace.ts`, `clients/typescript/src/client/git-client.ts` @@ -25,7 +28,8 @@ Routes: `GET /api/conversations/{runtime_conversation_id}/git/changes`, `GET /api/conversations/{runtime_conversation_id}/git/commits`, `GET /api/conversations/{runtime_conversation_id}/git/commits/{sha}/changes`, `GET /api/git/repositories/search`, `GET /api/git/changes`, -`GET /api/git/diff`, `GET /api/git/commits`, `GET /api/git/commits/{sha}/changes` +`GET /api/git/diff`, `GET /api/git/commits`, `GET /api/git/commits/{sha}/changes`, +`GET /api/git/repositories/{owner}/{repo}/pull-requests/{number}` ## Sub-features @@ -55,9 +59,16 @@ Routes: `GET /api/conversations/{runtime_conversation_id}/git/changes`, - `F15.repo-search-token`: without a stored token, GitHub search is 200 `{"items": [], "next_page_id": null, "missing_token": true}`; a secret named `github_token` or `GH_TOKEN` is picked up (a malformed `page_id` is then validated and refused with 400), and deleting it restores `missing_token`. - `F15.repo-search-errors`: `gitlab` and `bitbucket` are 400 (not supported), an unknown or missing `provider` is 422, and `limit` outside 1..100 is 422. - `F15.repo-search-unreachable`: when GitHub cannot be reached the search is 502 with the generic masked body and no token in it. -- `F15.repo-search-timeout`: when GitHub does not answer within 15 s the search is 504 with the masked body `GitHub repository search timed out` and no token in it. +- `F15.repo-search-timeout`: when GitHub does not answer within 15 s the search is 504 with the masked body `GitHub request timed out` and no token in it. - `F15.repo-search-github`: with a real GitHub token, search lists the token's repositories with a `page:offset` cursor and a case-insensitive `query` filter. - `F15.repo-search-rejected-token`: a provider's HTTP 401 for a rejected token is passed through as 401 with a generic error, without exposing the token. +- `F15.pr-status-token`: without a stored GitHub token, pull-request status is 409 with `detail.code` `missing_provider_token` (no request leaves the server). +- `F15.pr-status-errors`: `gitlab`, `bitbucket`, `azure_devops` and `bitbucket_data_center` are 400 (not supported, checked before the token); a missing or unknown `provider`, `number` below 1 or not an integer, an `owner` or `repo` of `.` or `..`, and an `owner` or `repo` outside `[A-Za-z0-9_.-]` are 422. +- `F15.pr-status-unreachable`: when GitHub cannot be reached, pull-request status is 502 with the masked body `GitHub request failed` and no token in it. +- `F15.pr-status-state`: against a local TLS stand-in for `api.github.com`, the status reports `open`, `closed`, `merged` and `draft`, `head_sha` (null without a head), and `checks` folded from check runs and commit statuses (`success`, `failure` wins over `pending`, `none` when there are none, `pending` when more than 500 check runs exist); the token is picked up under `GITHUB_TOKEN` or `github` and sent as a bearer token. +- `F15.pr-status-provider-errors`: GitHub's 404 stays 404, a rate limit (429, or 403 with `X-RateLimit-Remaining: 0`) is 429 with GitHub's `Retry-After`, a rejected token (401, other 403) is 502 `GitHub rejected the configured token`, 5xx (also from the check-runs call) is 502 and other 4xx 400, never exposing the token. +- `F15.pr-status-timeout`: when GitHub does not answer within 15 s the status is 504 with the masked body `GitHub request timed out`. +- `F15.pr-status-github`: with a real GitHub token, a real merged pull request reads back as `merged` with its 40-hex `head_sha` and a valid `checks` value. ## How to get to it (agent POV) @@ -79,6 +90,14 @@ Routes: `GET /api/conversations/{runtime_conversation_id}/git/changes`, `GET /api/git/repositories/search?provider=github&query=...&limit=1..100&page_id=...`; the token comes from the server's secrets store (`PUT /api/settings/secrets`, names `GITHUB_TOKEN`, `github_token`, `GH_TOKEN`, `github`, in that order). +- REST, pull-request status (global only): + `GET /api/git/repositories/{owner}/{repo}/pull-requests/{number}?provider=github` + returns `GitPullRequestStatus` (`git_provider`, `repository`, `number`, + `state` `open|closed|merged`, `draft`, `title`, `url`, `head_sha`, `checks` + `success|failure|pending|none`); same token lookup as search. No SDK or + TypeScript client method calls it yet. Driven by the `F15.pr-status-*` + bullets; the provider-dependent ones go through a local TLS stand-in for + `api.github.com` (translated), and `F15.pr-status-github` is the real one. - SDK: `RemoteWorkspace.git_changes(path)` and `git_diff(path)` (and the `AsyncRemoteWorkspace` versions) join a relative `path` to `working_dir` and call the global routes, or the scoped ones when the workspace has @@ -91,8 +110,8 @@ Routes: `GET /api/conversations/{runtime_conversation_id}/git/changes`, limit, pageId})`. Driven by `F15.ts-client` (a `node` program on `clients/typescript/dist`). - Agent Canvas (context only): the Changes tab (changes and diff), the - Commits tab (commits, commit changes, diff with `commit`) and the - repository picker (search). + Commits tab (commits, commit changes, diff with `commit`), the + repository picker (search) and pull-request status badges. ## Driving it with control-agent-server @@ -102,8 +121,9 @@ Preconditions: installed and `$DEEPSEEK_API_KEY` is set (only `F15.agent-changes` uses the model; only `F15.ts-client` uses `node`, and builds the TypeScript client with `npm ci && npm run build` if `clients/typescript/dist` is missing). No - launch flags are needed; `F15.repo-search-unreachable` restarts the run with - a dead proxy and restores it. + launch flags are needed; the `repo-search` and `pr-status` bullets that + need GitHub to fail or answer restart the run with a dead, silent or + stand-in HTTPS proxy and restore it (`restart --reset-config`). - The block below activates `deepseek-flash`, points `git` at the run's private config (author `QA Verify`, so commits made here need no global git identity), creates the `qa-f15-repo` fixture (commits `Initial commit` = @@ -700,10 +720,10 @@ Preconditions: call to `api.github.com` fails on any machine; restore afterwards. ```sh control-agent-server api PUT /api/settings/secrets --json '{"name": "github_token", "value": "qa-f15-not-a-real-token"}' --expect 200 - control-agent-server restart --env HTTPS_PROXY=http://127.0.0.1:9 --env https_proxy=http://127.0.0.1:9 \ + control-agent-server restart --env LITELLM_LOCAL_MODEL_COST_MAP=True --env HTTPS_PROXY=http://127.0.0.1:9 --env https_proxy=http://127.0.0.1:9 \ --env NO_PROXY=localhost --env no_proxy=localhost control-agent-server api GET /api/git/repositories/search --query provider=github --expect 502 \ - --check detail eq 'Internal Server Error' --check exception eq '502: GitHub repository search failed' \ + --check detail eq 'Internal Server Error' --check exception eq '502: GitHub request failed' \ --check . not-contains qa-f15-not-a-real-token --save F15.repo-search-unreachable/dead-proxy control-agent-server api DELETE /api/settings/secrets/github_token --expect 200 control-agent-server restart --reset-config @@ -711,7 +731,7 @@ Preconditions: control-agent-server doctor ``` 502 with the server's masked 5xx body - (`{"detail": "Internal Server Error", "exception": "502: GitHub repository search failed"}`), + (`{"detail": "Internal Server Error", "exception": "502: GitHub request failed"}`), without the token. `restart --reset-config` drops the dead proxy again (the run has no launch flags to lose), and the search is back to `missing_token` once the secret is deleted. @@ -727,10 +747,10 @@ Preconditions: for i in $(seq 50); do test -s "$F/hang-port" && break; sleep 0.1; done HANG="http://127.0.0.1:$(cat "$F/hang-port")" control-agent-server api PUT /api/settings/secrets --json '{"name": "github_token", "value": "qa-f15-not-a-real-token"}' --expect 200 - control-agent-server restart --env HTTPS_PROXY="$HANG" --env https_proxy="$HANG" --env NO_PROXY=localhost --env no_proxy=localhost + control-agent-server restart --env LITELLM_LOCAL_MODEL_COST_MAP=True --env HTTPS_PROXY="$HANG" --env https_proxy="$HANG" --env NO_PROXY=localhost --env no_proxy=localhost T0=$SECONDS control-agent-server api GET /api/git/repositories/search --query provider=github --expect 504 --expect-max-ms 30000 \ - --check detail eq 'Internal Server Error' --check exception eq '504: GitHub repository search timed out' \ + --check detail eq 'Internal Server Error' --check exception eq '504: GitHub request timed out' \ --check . not-contains qa-f15-not-a-real-token --save F15.repo-search-timeout/silent-proxy test $((SECONDS - T0)) -ge 14 kill "$HANG_PID" @@ -739,7 +759,7 @@ Preconditions: control-agent-server doctor ``` 504 after about 15 s (between 14 and 30), with the masked 5xx body - `{"detail": "Internal Server Error", "exception": "504: GitHub repository search timed out"}` + `{"detail": "Internal Server Error", "exception": "504: GitHub request timed out"}` and no token in it. - **Live GitHub listing (`F15.repo-search-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`.** Needs: `network` @@ -827,12 +847,12 @@ Preconditions: for i in $(seq 1 100); do test -s "$F/reject-port" && break; sleep 0.1; done REJECT_PROXY="http://127.0.0.1:$(cat "$F/reject-port")" control-agent-server api PUT /api/settings/secrets --json '{"name":"github_token","value":"qa-f15-rejected-token"}' --expect 200 - control-agent-server restart --env HTTPS_PROXY="$REJECT_PROXY" --env https_proxy="$REJECT_PROXY" \ + control-agent-server restart --env LITELLM_LOCAL_MODEL_COST_MAP=True --env HTTPS_PROXY="$REJECT_PROXY" --env https_proxy="$REJECT_PROXY" \ --env HTTP_PROXY="$REJECT_PROXY" --env http_proxy="$REJECT_PROXY" \ --env "SSL_CERT_FILE=$F/reject-cert.pem" --env NO_PROXY=localhost,127.0.0.1 --env no_proxy=localhost,127.0.0.1 control-agent-server doctor control-agent-server api GET /api/git/repositories/search --query provider=github --expect 401 \ - --check detail eq 'GitHub repository search failed' --check . not-contains qa-f15-rejected-token --save F15.repo-search-rejected-token/rejected + --check detail eq 'GitHub request failed' --check . not-contains qa-f15-rejected-token --save F15.repo-search-rejected-token/rejected control-agent-server assert --json "$F/reject-observed.json" --check request eq 'GET /user/repos' \ --check fake_token_received eq true --check returned_status eq 401 cp "$F/reject-observed.json" "$AGENT_SERVER_VERIFY_RUN/evidence/F15.repo-search-rejected-token/provider.json" @@ -848,8 +868,322 @@ Preconditions: generic error. Source policy also preserves 403, maps other 4xx to 400 and 5xx to 502; connection failure and timeout are covered above. +- **Pull-request status without a token (`F15.pr-status-token`).** No GitHub + secret is stored at this point (the search bullets deleted theirs), so the + route answers before any network call. + ```sh + control-agent-server api GET /api/git/repositories/search --query provider=github --expect 200 --check missing_token eq true + control-agent-server api GET /api/git/repositories/qa/f15/pull-requests/1 --query provider=github --expect 409 \ + --check detail.code eq missing_provider_token --check detail.message eq 'No github token is configured' \ + --save F15.pr-status-token/missing + ``` + 409 with `{"detail": {"code": "missing_provider_token", "message": "No github token is configured"}}`, + so a client can hide the badge instead of showing an error. +- **Pull-request status validation (`F15.pr-status-errors`).** Still no token: + every refusal here comes before the token lookup. `.` and `..` are sent + percent-encoded, because an HTTP client collapses a literal `..` segment + before sending (the request would then be a 404 on another path). + ```sh + P=/api/git/repositories/qa/f15/pull-requests + control-agent-server api GET "$P/1" --query provider=gitlab --expect 400 \ + --check detail eq "Pull requests are not supported for provider 'gitlab'" --save F15.pr-status-errors/gitlab + for PROVIDER in bitbucket azure_devops bitbucket_data_center; do + control-agent-server api GET "$P/1" --query provider="$PROVIDER" --expect 400 \ + --check detail eq "Pull requests are not supported for provider '$PROVIDER'" + done + control-agent-server api GET "$P/1" --expect 422 --check detail.0.loc eq '["query", "provider"]' --save F15.pr-status-errors/no-provider + control-agent-server api GET "$P/1" --query provider=qa-nope --expect 422 --check detail.0.type eq enum + control-agent-server api GET "$P/0" --query provider=github --expect 422 \ + --check detail.0.loc eq '["path", "number"]' --check detail.0.type eq greater_than_equal --save F15.pr-status-errors/number-0 + control-agent-server api GET "$P/-1" --query provider=github --expect 422 --check detail.0.type eq greater_than_equal + control-agent-server api GET "$P/x" --query provider=github --expect 422 --check detail.0.type eq int_parsing + control-agent-server api GET /api/git/repositories/%2E%2E/f15/pull-requests/1 --query provider=github --expect 422 \ + --check detail eq "'owner' and 'repo' must not be '.' or '..'" --save F15.pr-status-errors/dot-dot-owner + control-agent-server api GET /api/git/repositories/qa/%2E/pull-requests/1 --query provider=github --expect 422 \ + --check detail eq "'owner' and 'repo' must not be '.' or '..'" + control-agent-server api GET '/api/git/repositories/qa~x/f15/pull-requests/1' --query provider=github --expect 422 \ + --check detail.0.loc eq '["path", "owner"]' --check detail.0.type eq string_pattern_mismatch --save F15.pr-status-errors/bad-owner + control-agent-server api GET /api/git/repositories/qa/f15%20x/pull-requests/1 --query provider=github --expect 422 \ + --check detail.0.loc eq '["path", "repo"]' --check detail.0.type eq string_pattern_mismatch + control-agent-server api GET "$P/1" --query provider=github --expect 409 --check detail.code eq missing_provider_token + ``` + Unsupported providers are 400 with the provider named; FastAPI's 422 names + the offending `loc`; the dot segments get the handler's own 422 string. The + last call is the positive control: the same path with `github` reaches the + token lookup. +- **GitHub unreachable for pull-request status (`F15.pr-status-unreachable`).** + The `F15.repo-search-unreachable` dead proxy, for this route. + ```sh + control-agent-server api PUT /api/settings/secrets --json '{"name": "GITHUB_TOKEN", "value": "qa-f15-pr-token"}' --expect 200 + control-agent-server restart --env LITELLM_LOCAL_MODEL_COST_MAP=True --env HTTPS_PROXY=http://127.0.0.1:9 --env https_proxy=http://127.0.0.1:9 \ + --env NO_PROXY=localhost --env no_proxy=localhost + control-agent-server api GET /api/git/repositories/qa/f15/pull-requests/1 --query provider=github --expect 502 \ + --check detail eq 'Internal Server Error' --check exception eq '502: GitHub request failed' \ + --check . not-contains qa-f15-pr-token --save F15.pr-status-unreachable/dead-proxy + control-agent-server api DELETE /api/settings/secrets/GITHUB_TOKEN --expect 200 + control-agent-server restart --reset-config + control-agent-server api GET /api/git/repositories/qa/f15/pull-requests/1 --query provider=github --expect 409 \ + --check detail.code eq missing_provider_token + control-agent-server doctor + ``` + 502 with the masked 5xx body + (`{"detail": "Internal Server Error", "exception": "502: GitHub request failed"}`) + and no token; 409 again once the secret is deleted and the proxy dropped. +- **Pull-request state and CI result (`F15.pr-status-state`).** Translated: a + local TLS stand-in for `api.github.com` (an owned CONNECT proxy with a + self-signed certificate the server trusts through `SSL_CERT_FILE`, as in + `F15.repo-search-rejected-token`) serves scripted pull requests under owner + `qa`, one repository per scenario, and records what the server asked for in + `pr-observed.json`. It runs as the helper `qa-f15-github`, which the next two + bullets reuse; `F15.pr-status-timeout` stops it. + ```sh + cat > "$F/pr-github.py" <<'PY' + import datetime, json, pathlib, socketserver, ssl, sys, threading, time + from cryptography import x509 + from cryptography.hazmat.primitives import hashes, serialization + from cryptography.hazmat.primitives.asymmetric import rsa + from cryptography.x509.oid import NameOID + root = pathlib.Path(sys.argv[1]) + key = rsa.generate_private_key(public_exponent=65537, key_size=2048) + name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, 'api.github.com')]) + now = datetime.datetime.now(datetime.timezone.utc) + cert = (x509.CertificateBuilder().subject_name(name).issuer_name(name) + .public_key(key.public_key()).serial_number(x509.random_serial_number()) + .not_valid_before(now - datetime.timedelta(minutes=1)) + .not_valid_after(now + datetime.timedelta(days=1)) + .add_extension(x509.SubjectAlternativeName([x509.DNSName('api.github.com')]), critical=False) + .add_extension(x509.BasicConstraints(ca=True, path_length=None), critical=True) + .sign(key, hashes.SHA256())) + (root / 'pr-cert.pem').write_bytes(cert.public_bytes(serialization.Encoding.PEM)) + (root / 'pr-key.pem').write_bytes(key.private_bytes(serialization.Encoding.PEM, + serialization.PrivateFormat.PKCS8, serialization.NoEncryption())) + context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + context.load_cert_chain(root / 'pr-cert.pem', root / 'pr-key.pem') + SHA = {'ok': 'a' * 40, 'merged': 'b' * 40, 'closed': 'c' * 40, 'draft': 'd' * 40, + 'status-fail': 'e' * 40, 'many': 'f' * 40, 'checks-boom': '1' * 40} + def pr(repo, number, **extra): + body = {'number': number, 'state': 'open', 'draft': False, 'merged': False, 'merged_at': None, + 'title': f'QA {repo}', 'html_url': f'https://github.com/qa/{repo}/pull/{number}', + 'head': {'sha': SHA.get(repo)}} + body.update(extra) + return body + PULLS = {'ok': pr('ok', 1), 'merged': pr('merged', 2, state='closed', merged=True, merged_at='2026-11-10T00:00:00Z'), + 'closed': pr('closed', 3, state='closed'), 'draft': pr('draft', 4, draft=True), + 'status-fail': pr('status-fail', 5), 'nohead': pr('nohead', 6, head={}), 'many': pr('many', 7), + 'checks-boom': pr('checks-boom', 8)} + def run(status='completed', conclusion='success'): + return {'status': status, 'conclusion': conclusion} + RUNS = {'ok': [run(), run(conclusion='skipped')], 'merged': [], 'closed': [run(), run(conclusion='failure')], + 'draft': [run(status='in_progress', conclusion=None)], 'status-fail': [run(status='queued', conclusion=None)]} + STATUSES = {'ok': [{'state': 'success'}], 'merged': [], 'closed': [], 'draft': [{'state': 'success'}], + 'status-fail': [{'state': 'success'}, {'state': 'error'}], 'many': [], 'checks-boom': []} + ERRORS = {'gone': (404, {}), 'badtoken': (401, {}), 'forbidden': (403, {}), + 'ratelimit': (403, {'X-RateLimit-Remaining': '0', 'Retry-After': '42'}), + 'toomany': (429, {'Retry-After': '7'}), 'boom': (500, {}), 'unproc': (422, {})} + lock = threading.Lock() + observed = {'requests': 0, 'bad_auth': 0, 'api_version': None, 'accept': None, 'count': {}, 'many_pages': []} + def record(repo, kind, headers, query): + with lock: + observed['requests'] += 1 + if headers.get('authorization') != 'Bearer qa-f15-pr-token': + observed['bad_auth'] += 1 + observed['api_version'] = headers.get('x-github-api-version') + observed['accept'] = headers.get('accept') + per = observed['count'].setdefault(repo, {}) + per[kind] = per.get(kind, 0) + 1 + if repo == 'many' and kind == 'check_runs': + observed['many_pages'].append(query) + tmp = root / 'pr-observed.json.tmp' + tmp.write_text(json.dumps(observed, indent=1, sort_keys=True)) + tmp.replace(root / 'pr-observed.json') + def read_head(conn): + value = b'' + while b'\r\n\r\n' not in value: + chunk = conn.recv(4096) + if not chunk: raise ConnectionError('incomplete request') + value += chunk + lines = value.decode('latin1').split('\r\n') + headers = {} + for line in lines[1:]: + if ':' in line: + k, v = line.split(':', 1) + headers[k.strip().lower()] = v.strip() + return lines[0], headers + def reply(conn, status, body, headers=None): + data = json.dumps(body).encode() + extra = ''.join(f'{k}: {v}\r\n' for k, v in (headers or {}).items()) + conn.sendall(f'HTTP/1.1 {status} QA\r\nContent-Type: application/json\r\nContent-Length: {len(data)}\r\n{extra}Connection: close\r\n\r\n'.encode() + data) + def answer(conn, line, headers): + method, target, _ = line.split(' ', 2) + path, _, query = target.partition('?') + parts = path.strip('/').split('/') + if method != 'GET' or len(parts) < 5 or parts[:2] != ['repos', 'qa']: + return reply(conn, 404, {'message': 'Not Found'}) + repo = parts[2] + if parts[3] == 'pulls': + record(repo, 'pulls', headers, query) + if repo == 'slow': + time.sleep(20) + if repo in ERRORS: + status, extra = ERRORS[repo] + return reply(conn, status, {'message': 'qa'}, extra) + if repo in PULLS: + return reply(conn, 200, PULLS[repo]) + return reply(conn, 404, {'message': 'Not Found'}) + if parts[3] == 'commits' and parts[-1] == 'check-runs': + record(repo, 'check_runs', headers, query) + if repo == 'checks-boom': + return reply(conn, 500, {'message': 'qa'}) + if repo == 'many': + return reply(conn, 200, {'total_count': 600, 'check_runs': [run() for _ in range(100)]}) + runs = RUNS.get(repo, []) + return reply(conn, 200, {'total_count': len(runs), 'check_runs': runs}) + if parts[3] == 'commits' and parts[-1] == 'status': + record(repo, 'status', headers, query) + return reply(conn, 200, {'state': 'pending', 'statuses': STATUSES.get(repo, [])}) + return reply(conn, 404, {'message': 'Not Found'}) + class Handler(socketserver.BaseRequestHandler): + def handle(self): + line, _ = read_head(self.request) + if not line.startswith('CONNECT api.github.com:443 '): + self.request.sendall(b'HTTP/1.1 502 Bad Gateway\r\nContent-Length: 0\r\nConnection: close\r\n\r\n') + return + self.request.sendall(b'HTTP/1.1 200 Connection Established\r\n\r\n') + try: + with context.wrap_socket(self.request, server_side=True) as conn: + answer(conn, *read_head(conn)) + except (OSError, ConnectionError): + pass + class Server(socketserver.ThreadingTCPServer): + daemon_threads = True + with Server(('127.0.0.1', 0), Handler) as server: + (root / 'pr-port').write_text(str(server.server_address[1])) + server.serve_forever() + PY + rm -f "$F/pr-port" "$F/pr-observed.json" + control-agent-server helper start --name qa-f15-github -- uv run --quiet python "$F/pr-github.py" "$F" + for i in $(seq 1 100); do test -s "$F/pr-port" && break; sleep 0.1; done + PR_PROXY="http://127.0.0.1:$(cat "$F/pr-port")" + control-agent-server api PUT /api/settings/secrets --json '{"name": "GITHUB_TOKEN", "value": "qa-f15-pr-token"}' --expect 200 + control-agent-server restart --env LITELLM_LOCAL_MODEL_COST_MAP=True --env HTTPS_PROXY="$PR_PROXY" --env https_proxy="$PR_PROXY" \ + --env HTTP_PROXY="$PR_PROXY" --env http_proxy="$PR_PROXY" \ + --env "SSL_CERT_FILE=$F/pr-cert.pem" --env NO_PROXY=localhost,127.0.0.1 --env no_proxy=localhost,127.0.0.1 + control-agent-server doctor + PR=/api/git/repositories/qa + control-agent-server api GET "$PR/ok/pull-requests/1" --query provider=github --expect 200 \ + --check git_provider eq github --check repository eq qa/ok --check number eq 1 --check state eq open \ + --check draft eq false --check title eq 'QA ok' --check url eq https://github.com/qa/ok/pull/1 \ + --check head_sha eq aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa --check checks eq success --save F15.pr-status-state/open-success + control-agent-server api GET "$PR/merged/pull-requests/2" --query provider=github --expect 200 \ + --check state eq merged --check checks eq none --save F15.pr-status-state/merged-none + control-agent-server api GET "$PR/closed/pull-requests/3" --query provider=github --expect 200 \ + --check state eq closed --check checks eq failure --save F15.pr-status-state/closed-failure + control-agent-server api GET "$PR/draft/pull-requests/4" --query provider=github --expect 200 \ + --check state eq open --check draft eq true --check checks eq pending --save F15.pr-status-state/draft-pending + control-agent-server api GET "$PR/status-fail/pull-requests/5" --query provider=github --expect 200 \ + --check checks eq failure --save F15.pr-status-state/status-error-wins + control-agent-server api GET "$PR/nohead/pull-requests/6" --query provider=github --expect 200 \ + --check head_sha eq null --check checks eq none --save F15.pr-status-state/no-head + control-agent-server api GET "$PR/many/pull-requests/7" --query provider=github --expect 200 \ + --check checks eq pending --save F15.pr-status-state/too-many-check-runs + control-agent-server assert --json "$F/pr-observed.json" --check bad_auth eq 0 --check api_version eq 2022-11-28 \ + --check accept eq application/vnd.github+json --check count.ok.pulls eq 1 --check count.ok.check_runs eq 1 \ + --check count.ok.status eq 1 --check count.nohead.pulls eq 1 --check count.nohead.check_runs missing \ + --check count.many.check_runs eq 5 --check many_pages.0 eq 'per_page=100&page=1' --check many_pages.4 eq 'per_page=100&page=5' + cp "$F/pr-observed.json" "$AGENT_SERVER_VERIFY_RUN/evidence/F15.pr-status-state/provider.json" + control-agent-server api DELETE /api/settings/secrets/GITHUB_TOKEN --expect 200 + control-agent-server api GET "$PR/ok/pull-requests/1" --query provider=github --expect 409 --check detail.code eq missing_provider_token + control-agent-server api PUT /api/settings/secrets --json '{"name": "github", "value": "qa-f15-pr-token"}' --expect 200 + control-agent-server api GET "$PR/ok/pull-requests/1" --query provider=github --expect 200 --check checks eq success \ + --save F15.pr-status-state/token-name-github + control-agent-server assert --json "$F/pr-observed.json" --check bad_auth eq 0 --check count.ok.pulls eq 2 + ``` + Each scenario's `state`, `draft`, `head_sha` and `checks`: `success` from + completed successful or skipped runs plus a successful status, `none` with + neither, `failure` from a failed run or an `error` status (even beside a + queued run), `pending` from an in-progress run, and `pending` when GitHub + reports 600 runs (the server reads five pages of 100, then stops). Without + a head SHA no commit endpoints are called. Every request carried the stored + token as `Bearer`, `X-GitHub-Api-Version: 2022-11-28` and the GitHub JSON + `Accept`; deleting `GITHUB_TOKEN` gives 409 although the stand-in is still + up, and the lowest-priority name `github` is picked up again. The `github` + secret stays for the next two bullets. +- **Provider errors for pull-request status (`F15.pr-status-provider-errors`).** + The same stand-in answers per repository with GitHub's error statuses. + Requires: `F15.pr-status-state` + ```sh + control-agent-server api GET "$PR/gone/pull-requests/1" --query provider=github --expect 404 \ + --check detail eq 'GitHub did not find it' --save F15.pr-status-provider-errors/not-found + control-agent-server api GET "$PR/ratelimit/pull-requests/1" --query provider=github --expect 429 \ + --check detail eq 'GitHub rate limit reached' --check-header retry-after eq 42 --save F15.pr-status-provider-errors/rate-limit-403 + control-agent-server api GET "$PR/toomany/pull-requests/1" --query provider=github --expect 429 \ + --check detail eq 'GitHub rate limit reached' --check-header retry-after eq 7 --save F15.pr-status-provider-errors/rate-limit-429 + control-agent-server api GET "$PR/badtoken/pull-requests/1" --query provider=github --expect 502 \ + --check detail eq 'Internal Server Error' --check exception eq '502: GitHub rejected the configured token' \ + --check . not-contains qa-f15-pr-token --save F15.pr-status-provider-errors/rejected-401 + control-agent-server api GET "$PR/forbidden/pull-requests/1" --query provider=github --expect 502 \ + --check exception eq '502: GitHub rejected the configured token' --check . not-contains qa-f15-pr-token + control-agent-server api GET "$PR/boom/pull-requests/1" --query provider=github --expect 502 \ + --check detail eq 'Internal Server Error' --check exception eq '502: GitHub request failed' --save F15.pr-status-provider-errors/server-error + control-agent-server api GET "$PR/checks-boom/pull-requests/8" --query provider=github --expect 502 \ + --check exception eq '502: GitHub request failed' --save F15.pr-status-provider-errors/check-runs-error + control-agent-server api GET "$PR/unproc/pull-requests/1" --query provider=github --expect 400 \ + --check detail eq 'GitHub request failed' --check . not-contains qa-f15-pr-token --save F15.pr-status-provider-errors/other-4xx + control-agent-server assert --json "$F/pr-observed.json" --check bad_auth eq 0 --check count.gone.pulls eq 1 \ + --check count.checks-boom.check_runs eq 1 --check count.boom.check_runs missing + ``` + 404 stays 404 (`GitHub did not find it`); a 403 with + `X-RateLimit-Remaining: 0` and a 429 both become 429 and carry GitHub's + `Retry-After` (42 and 7); 401 and a plain 403 become 502 so a client never + mistakes them for its own session key being wrong; GitHub 500 (on the pull + request or on its check runs) is 502 and 422 is 400. The 5xx bodies are + masked (`Internal Server Error`, reason in `exception`), and none contains + the token. +- **Pull-request status timeout (`F15.pr-status-timeout`).** The stand-in holds + the `qa/slow` pull request for 20 s, so the server's own 15 s timeout fires + (the timer is under test, so the bullet measures the wait). Then the + stand-in, the secret and the proxy settings are removed. + Requires: `F15.pr-status-state` + ```sh + T0=$SECONDS + control-agent-server api GET "$PR/slow/pull-requests/1" --query provider=github --expect 504 --expect-max-ms 30000 \ + --check detail eq 'Internal Server Error' --check exception eq '504: GitHub request timed out' \ + --check . not-contains qa-f15-pr-token --save F15.pr-status-timeout/slow + test $((SECONDS - T0)) -ge 14 + control-agent-server helper stop qa-f15-github + control-agent-server api DELETE /api/settings/secrets/github --expect 200 + control-agent-server restart --reset-config + control-agent-server api GET "$PR/ok/pull-requests/1" --query provider=github --expect 409 --check detail.code eq missing_provider_token + control-agent-server doctor + ``` + 504 after about 15 s (between 14 and 30) with + `{"detail": "Internal Server Error", "exception": "504: GitHub request timed out"}`; + afterwards the run is back to no token and no proxy (409). +- **Live pull-request status (`F15.pr-status-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`.** + Needs: `network` + + Set `$QA_GITHUB_TOKEN` to a GitHub token that can read the public + `OpenHands/agent-sdk` repository. Pull request 6704 (the one that added this + route) is merged, so its state is fixed; its CI result is whatever GitHub + reports, so only the value set is checked. The token travels through stdin + and is registered with the redactor as in `F15.repo-search-github`. + ```sh + : "${QA_GITHUB_TOKEN:?Set a repository-read token before running this recipe}" + trap 'control-agent-server api DELETE /api/settings/secrets/github_token --expect 200 >/dev/null 2>&1 || true' EXIT + python3 -c 'import json, os; print(json.dumps({"name":"github_token","value":os.environ["QA_GITHUB_TOKEN"]}))' \ + | env OPENHANDS_API_KEY="$QA_GITHUB_TOKEN" control-agent-server api PUT /api/settings/secrets --stdin --expect 200 --quiet + control-agent-server api GET /api/git/repositories/OpenHands/agent-sdk/pull-requests/6704 --query provider=github --expect 200 \ + --check state eq merged --check number eq 6704 --check repository eq OpenHands/agent-sdk \ + --check head_sha matches '^[0-9a-f]{40}$' --check checks matches '^(success|failure|pending|none)$' \ + --check url eq https://github.com/OpenHands/agent-sdk/pull/6704 --save F15.pr-status-github/merged + control-agent-server api DELETE /api/settings/secrets/github_token --expect 200 + trap - EXIT + ``` + `merged`, a 40-hex head SHA and one of the four `checks` values. + ## Gotchas +- Agent Canvas does not call the pull-request status route yet, and it pins Agent Server 1.61.0, which predates it; the Canvas follow-up is [OpenHands/OpenHands#19644](https://github.com/OpenHands/OpenHands/issues/19644). - `changes` and `diff` without `ref` use the *display* base policy (`get_valid_ref(..., purpose="display")`), which differs from the *export* policy of `GET /api/file/archive` (`git-delta`): on a remote-less fixture @@ -898,6 +1232,24 @@ Preconditions: Statuses from GitHub: 401 and 403 pass through, other 4xx are 400, 5xx and transport errors 502 (masked as `Internal Server Error` with the reason in `exception`), a timeout 504. +- Pull-request status shares the token lookup with search but not its error + mapping: GitHub's 404 stays 404, a rate limit (429, or 403 with + `X-RateLimit-Remaining: 0`) is 429 with `Retry-After` passed through, and a + rejected token (401, other 403) is 502, never 401, so a client cannot + mistake it for its own session key. No token is a 409 with + `detail.code` `missing_provider_token`, not an empty page. Both provider + routes now share the messages `GitHub request failed` and + `GitHub request timed out` (search said `GitHub repository search ...` + before #6704). +- A literal `.` or `..` path segment never reaches the pull-request route: + the HTTP client normalizes it away first. Send `%2E` / `%2E%2E` to reach the + handler's own 422. +- Every restart that points `HTTPS_PROXY` at a dead, silent or stand-in proxy + also sets `LITELLM_LOCAL_MODEL_COST_MAP=True`. Otherwise `import litellm` + tries to fetch its model cost map through that proxy at startup, fails, and + can intermittently crash the server with a `_DeadlockError` on + `litellm.rust_bridge.catalog` (seen once on the stand-in proxy: + `Server exited with code 1 during startup`). - `restart --env` values persist for later restarts of the run; clear them with `restart --reset-config` as `F15.repo-search-unreachable` does (that also drops `Launch:` flags, which this family has none of). diff --git a/.agents/skills/verify-agent-server/references/feature-map/F18-settings-and-secrets.md b/.agents/skills/verify-agent-server/references/feature-map/F18-settings-and-secrets.md index ca176d105ea..62138799630 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F18-settings-and-secrets.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F18-settings-and-secrets.md @@ -15,7 +15,7 @@ raw value as `text/plain` (the target of the SDK's `LookupSecret` URLs), and cipher (`OH_SECRET_KEY`, falling back to the session key) and survive restarts. Settings changes emit no WebSocket events. -Source: `openhands-agent-server/openhands/agent_server/settings_router.py`, `openhands-agent-server/openhands/agent_server/_secrets_exposure.py`, `openhands-agent-server/openhands/agent_server/persistence/models.py`, `openhands-agent-server/openhands/agent_server/persistence/store.py`, `openhands-sdk/openhands/sdk/settings/api_models.py`, `openhands-sdk/openhands/sdk/settings/model.py`, `openhands-sdk/openhands/sdk/utils/pydantic_secrets.py`, `openhands-sdk/openhands/sdk/utils/cipher.py`, `openhands-sdk/openhands/sdk/workspace/remote/base.py`, `openhands-sdk/openhands/sdk/workspace/repo.py`, `clients/typescript/src/client/settings-client.ts` +Source: `openhands-agent-server/openhands/agent_server/settings_router.py`, `openhands-agent-server/openhands/agent_server/experimental_features_router.py`, `openhands-agent-server/openhands/agent_server/_secrets_exposure.py`, `openhands-agent-server/openhands/agent_server/persistence/models.py`, `openhands-agent-server/openhands/agent_server/persistence/store.py`, `openhands-sdk/openhands/sdk/settings/api_models.py`, `openhands-sdk/openhands/sdk/settings/model.py`, `openhands-sdk/openhands/sdk/utils/pydantic_secrets.py`, `openhands-sdk/openhands/sdk/utils/cipher.py`, `openhands-sdk/openhands/sdk/workspace/remote/base.py`, `openhands-sdk/openhands/sdk/workspace/repo.py`, `clients/typescript/src/client/settings-client.ts` Needs: `git`, `node` @@ -49,9 +49,9 @@ Routes: `GET /api/settings/agent-schema`, `GET /api/settings/conversation-schema - `F18.sdk-clone-repos`: with the server secret `GITHUB_TOKEN` stored, the SDK's `RemoteWorkspace.clone_repos()` reads it from `GET /api/settings/secrets/GITHUB_TOKEN` and clones with it (a local git remote reachable only through the token-bearing URL), a provider whose token secret is missing (`GITLAB_TOKEN`) clones anonymously, and `get_repos_context()` names both clones. - `F18.ts-client`: the TypeScript `SettingsClient` reads both schemas, reads settings redacted, `plaintext` and `encrypted`, applies a sparse `updateSettings` that a second read shows, upserts, lists (also with `agentProfileId`), reads as text and deletes a secret, and rejects an unknown secret (404), a bad name (422) and a wrong key (401) with an `HttpError`. - `F18.persist-restart`: settings (plaintext key, model, conversation settings) and secrets read back unchanged after a restart with the same cipher key. -- `F18.experimental-features`: `GET /api/settings/experimental` lists the experimental features the user can switch (`conversation_reactions`, off) without creating `settings.json`, and `PATCH /api/settings/experimental/{key}` answers 404 for a key the server does not know and stores nothing. +- `F18.experimental-features`: `GET /api/settings/experimental` lists the experimental features the user can switch (`conversation_reactions`, off) without writing `settings.json`, and `PATCH /api/settings/experimental/{key}` answers 404 for a key the server does not know and stores nothing (`experimental_features` stays `{}`). - `F18.upgrade-from-release`: settings, an LLM key and a secret saved by the latest released server read back identical after the same state is restarted on this checkout (`restart --checkout`), the upgrade does not rewrite either file, and the experimental routes answer. -- `F18.corrupted-files`: with an unparsable `settings.json`, GET silently returns defaults and PATCH is 409 without touching the file; with an unparsable `secrets.json` the list is empty, GET by name 404, PUT and DELETE 500; restoring the files restores the values. +- `F18.corrupted-files`: with an unparsable `settings.json`, GET silently returns defaults (every experimental feature off) and PATCH, of the settings or of an experimental feature, is 409 without touching the file; with an unparsable `secrets.json` the list is empty, GET by name 404, PUT and DELETE 500; restoring the files restores the values. - `F18.no-cipher`: without any cipher key, `X-Expose-Secrets: encrypted` is 503, `plaintext` still works, both files store plaintext, and the secrets routes are open without a key. - `F18.secret-empty-value`: a PUT whose value is empty, blank or the redaction placeholder is refused with 422 and neither wipes an existing value nor leaves a listed but unreadable secret (fixed by #5682). - `F18.dangling-pointers`: PATCH refuses an `active_agent_profile_id` or `active_meta_profile` that names nothing with 404, as the dedicated activate routes do, and stores nothing (fixed by #5847). @@ -71,6 +71,12 @@ Routes: `GET /api/settings/agent-schema`, `GET /api/settings/conversation-schema `PUT /api/settings/secrets` `{"name", "value", "description"?}`, `GET /api/settings/secrets/{name}` (raw `text/plain`), `DELETE /api/settings/secrets/{name}`. +- REST: `GET /api/settings/experimental` (a list of + `{key, label, description, enabled}`) and + `PATCH /api/settings/experimental/{key}` `{"enabled": bool}` (404 for an + unknown key, 409 when `settings.json` cannot be read); the switches are + stored in `settings.json` `experimental_features` (what each one does is + mapped by its own family: `conversation_reactions` in F37). - Neighbours used as arrange steps and second views, owned by other families: `POST/DELETE /api/profiles/{name}` and `GET /api/profiles` (LLM profiles), `POST/GET/DELETE /api/agent-profiles/{name}` and @@ -96,7 +102,9 @@ Routes: `GET /api/settings/agent-schema`, `GET /api/settings/conversation-schema `getConversationSchema()`, `getSettings({exposeSecrets})`, `updateSettings(request)`, `listSecrets({agentProfileId})`, `upsertSecret(request)`, `getSecret(name)` (`responseType: 'text'`), - `deleteSecret(name)` (`clients/typescript/src/client/settings-client.ts`); + `deleteSecret(name)`, `listExperimentalFeatures()` and + `updateExperimentalFeature(key, enabled)` (driven by `F37.ts-client`) + (`clients/typescript/src/client/settings-client.ts`); refusals reject with `HttpError` (`isHttpError`, `status`, `detail`). Recipe `F18.ts-client` runs them with `node` on the built package (`clients/typescript/dist`). The same class's `createMcpServer`, @@ -668,28 +676,36 @@ Preconditions: - **Experimental features (`F18.experimental-features`).** List them and try to switch one the server does not know. ```sh + S="$AGENT_SERVER_VERIFY_RUN/home/.openhands/settings.json" + SUM=$({ cat "$S" 2>/dev/null || true; } | sha256sum) control-agent-server api GET /api/settings/experimental --expect 200 --check . len-eq 1 --check 0.key eq conversation_reactions \ - --check 0.enabled eq false --save F18.experimental-features/list + --check 0.label eq 'Conversation reactions' --check 0.description exists --check 0.enabled eq false --save F18.experimental-features/list control-agent-server api PATCH /api/settings/experimental/qa_f18_unknown --json '{"enabled": true}' --expect 404 \ --check detail eq 'Unknown experimental feature: qa_f18_unknown' --save F18.experimental-features/unknown control-agent-server api PATCH /api/settings/experimental/qa_f18_unknown --json '{}' --expect 422 control-agent-server api GET /api/settings/experimental --auth none --expect 401 - if grep -qs experimental_features "$AGENT_SERVER_VERIFY_RUN/home/.openhands/settings.json"; then false; fi + control-agent-server api PATCH /api/settings/experimental/conversation_reactions --auth none --json '{"enabled": true}' --expect 401 + test "$SUM" = "$({ cat "$S" 2>/dev/null || true; } | sha256sum)" + if test -e "$S"; then jq -e '.experimental_features == {}' "$S"; fi ``` The list holds `conversation_reactions`, off (F37 switches it), as `{key, label, description, enabled}`. A key the server does not know is a - 404, and the refused switch leaves `settings.json` without an - `experimental_features` entry. + 404, and neither the reads nor the refused switches create or change + `settings.json`; once other bullets saved it, its `experimental_features` + is the empty object every save writes. - **Upgrade from the latest release (`F18.upgrade-from-release`).** A second run starts on the newest release tag, saves settings as a user of that release would, and is restarted on this checkout over the same state. ```sh HERE="${AGENT_SERVER_VERIFY_CHECKOUT:-$PWD}" - TAG=$(git -C "$HERE" describe --tags --abbrev=0 --match 'v*') + TAG=$(git -C "$HERE" describe --tags --abbrev=0 --match 'v*' 2>/dev/null) || + TAG=$(git -C "$HERE" ls-remote --tags --refs --sort=-v:refname origin 'v*' | sed -n '1s|.*refs/tags/||p') + test -n "$TAG" REL="$F18/release-$TAG" - git -C "$HERE" worktree add --detach "$REL" "$TAG" >/dev/null 2>&1 + git -C "$HERE" worktree add --detach "$REL" "$TAG" >/dev/null 2>&1 || + git clone -q --depth 1 --branch "$TAG" "$(git -C "$HERE" remote get-url origin)" "$REL" UP=$(control-agent-server launch --new --name f18-upgrade --checkout "$REL" --timeout 900 --print-run) - trap 'control-agent-server stop --run "$UP" >/dev/null; git -C "$HERE" worktree remove --force "$REL"' EXIT + trap 'control-agent-server stop --run "$UP" >/dev/null; git -C "$HERE" worktree remove --force "$REL" 2>/dev/null || rm -rf "$REL"' EXIT control-agent-server api PATCH /api/settings --run "$UP" --json '{"agent_settings_diff": {"llm": {"model": "openai/qa-f18-upgrade", "api_key": "qa-f18-upgrade-key"}}, "conversation_settings_diff": {"max_iterations": 77}, "misc_settings_diff": {"qa_f18": "from-release"}}' --expect 200 control-agent-server api PUT /api/settings/secrets --run "$UP" --json '{"name": "QA_F18_UPGRADE", "value": "qa-f18-upgrade-secret"}' --expect 200 BEFORE=$(sha256sum "$UP/home/.openhands/settings.json" "$UP/home/.openhands/secrets.json") @@ -698,13 +714,17 @@ Preconditions: --check agent_settings.llm.model eq openai/qa-f18-upgrade --check agent_settings.llm.api_key eq qa-f18-upgrade-key \ --check conversation_settings.max_iterations eq 77 --check misc_settings.qa_f18 eq from-release --save F18.upgrade-from-release/settings control-agent-server api GET /api/settings/secrets/QA_F18_UPGRADE --run "$UP" --expect 200 --check . eq qa-f18-upgrade-secret - control-agent-server api GET /api/settings/experimental --run "$UP" --expect 200 --save F18.upgrade-from-release/experimental + control-agent-server api GET /api/settings/experimental --run "$UP" --expect 200 --check 0.key eq conversation_reactions \ + --check 0.enabled eq false --save F18.upgrade-from-release/experimental test "$BEFORE" = "$(sha256sum "$UP/home/.openhands/settings.json" "$UP/home/.openhands/secrets.json")" ``` Everything the release saved reads back unchanged on this checkout, and reading it does not rewrite `settings.json` or `secrets.json` (same checksums). The first launch of a release worktree builds its environment - with `uv`, which needs the package index and can take minutes. + with `uv`, which needs the package index and can take minutes. A shallow + clone without tags (a CI or cloud checkout) has no tag to describe or to + add as a worktree: the recipe then takes the newest `v*` tag from the + remote and clones it into the run (`network`). - **Corrupted files (`F18.corrupted-files`).** Back up both files, overwrite them with broken JSON, drive every route, restore. @@ -723,6 +743,9 @@ Preconditions: --check agent_settings.llm.model eq "$DEFAULT_MODEL" --save F18.corrupted-files/get-settings control-agent-server api PATCH /api/settings --json '{"misc_settings_diff": {"qa_f18_probe": 1}}' --expect 409 \ --check detail contains 'Settings file is corrupted or encrypted with a different key' --save F18.corrupted-files/patch + control-agent-server api GET /api/settings/experimental --expect 200 --check 0.enabled eq false + control-agent-server api PATCH /api/settings/experimental/conversation_reactions --json '{"enabled": true}' --expect 409 \ + --check detail eq 'Settings file is corrupted or encrypted with a different key' --save F18.corrupted-files/patch-experimental control-agent-server state cat home/.openhands/settings.json --contains qa-f18-corrupt control-agent-server api GET /api/settings/secrets --expect 200 --check secrets len-eq 0 --save F18.corrupted-files/list-secrets control-agent-server api GET /api/settings/secrets/QA_F18_TOKEN --expect 404 @@ -735,8 +758,9 @@ Preconditions: control-agent-server api GET /api/settings --header 'X-Expose-Secrets: plaintext' --check agent_settings.llm.api_key eq "$QA_F18_LLM_KEY" control-agent-server api GET /api/settings/secrets/QA_F18_TOKEN --expect 200 --check . eq "$QA_F18_SECRET" --save F18.corrupted-files/restored ``` - A broken `settings.json` reads as defaults (200, no error), and PATCH - refuses with 409 instead of overwriting it. A broken `secrets.json` lists + A broken `settings.json` reads as defaults (200, no error; every + experimental feature off), and both PATCH routes refuse with 409 instead + of overwriting it. A broken `secrets.json` lists nothing and hides every value (404); PUT and DELETE refuse with 500 (the reason is in `exception`). Neither file was touched, and restoring them restores the values without a restart. @@ -1039,3 +1063,8 @@ Preconditions: token back adds `--unsafe-unredacted` to its `--field` read (`TOKEN=$(... --field ... --unsafe-unredacted)`) and keeps it out of evidence. +- `experimental_features` lives inside `settings.json` schema v7 without a + version bump. A server older than the experimental routes ignores the key + and drops it on its next save, so restarting on such a release + (`restart --checkout`), saving any setting there and coming back turns + every feature off again (v1.62.0 is such a release). diff --git a/.agents/skills/verify-agent-server/references/feature-map/F21-agent-profiles.md b/.agents/skills/verify-agent-server/references/feature-map/F21-agent-profiles.md index c5f6c84ff71..35d02e2f869 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F21-agent-profiles.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F21-agent-profiles.md @@ -132,9 +132,9 @@ Preconditions: restarts the run twice (with `OH_ACP_SKILL_SOURCING=openhands_managed`, then with `--reset-config`). `F21.store-busy`, `F21.launch-store-busy` and `F21.busy-blocks-server` hold the store lock for about 30 seconds - each. The three known-bug bullets come last, then - `F21.json-suffix-identity`, which activates its own profile and ends by - deleting it, so the run is left with `default` active. + each. `F21.busy-blocks-server`, `F21.dangling-llm-report` and + `F21.ts-payload-types` come last, then `F21.json-suffix-identity`, + which activates its own profile and ends by deleting it, so the run is left with `default` active. ```sh control-agent-server llm preset deepseek @@ -974,8 +974,13 @@ printf -- '---\nname: qa-f21-skill\ndescription: QA F21 user skill\n---\nQA_F21_ refuses with 422 `extra_forbidden`), and `ACPAgentProfile` declares `acp_startup_timeout`. The client's unit test `agent-server-contract-types.test.ts` pins the same key sets against the - generated contract. `public_skills` is declared on `OpenHandsAgentProfile` and `ACPAgentProfile` - (#6187, which closed #6183). + generated contract. `public_skills` is declared on `OpenHandsAgentProfile` and `ACPAgentProfile`, + and `acp_effort` (which the server's ACP profile carries) on both the + generated `AcpAgentProfile` and the exported `ACPAgentProfile`. The + generated contract is regenerated only from the pinned release image, so + a field the server adds after the last release fails `contract.ts` with + `TS2353 ... does not exist` until the next release regenerates it; the + client's key-parity test requires the exported type to change with it. - **`.json` names keep their identity (`F21.json-suffix-identity`).** The store strips a trailing `.json` from the file name and from the profile's `name`, so `x.json` and `x` are one profile. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F24-openai-subscription.md b/.agents/skills/verify-agent-server/references/feature-map/F24-openai-subscription.md index 0e1915c67c2..51d13b2e725 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F24-openai-subscription.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F24-openai-subscription.md @@ -532,6 +532,7 @@ Preconditions: ## Gotchas +- `F24.offline-safe` starts a server whose `HTTPS_PROXY` refuses every connection, so LiteLLM's model cost-map fetch fails at import. Under load that startup can crash with an `ImportError` or `DeadlockError` in `litellm.rust_bridge.catalog` (#6749); the bullet then fails at `launch`, and a rerun on a quiet machine passes. - The models route returns `OPENAI_CODEX_MODELS` (`openhands-sdk/openhands/sdk/llm/auth/openai.py`), not the `codex` ACP provider's deprecated `available_models`. The two lists matched until #5436 diff --git a/.agents/skills/verify-agent-server/references/feature-map/F25-skills.md b/.agents/skills/verify-agent-server/references/feature-map/F25-skills.md index e7483938aa7..6675e5bb7a5 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F25-skills.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F25-skills.md @@ -59,6 +59,7 @@ Routes: `POST /api/skills`, `GET /api/skills/catalog`, `POST /api/skills/sync`, - `F25.ts-client`: the TypeScript `SkillsClient` installs a skill from a git source, lists, reads, disables and re-enables it (`toggleSkill`, with `getSkills` following the flag), refreshes, syncs, reads the marketplace and uninstalls it; an unknown skill is a 404 `HttpError` and a client without a key gets 401. - `F25.marketplace-stale-cache`: after a sync brings a new entry into the public repository, the catalog lists it: a successful sync clears the catalog's 5-minute cache, which never keeps an empty result. - `F25.uninstall`: uninstall deletes the directory and the metadata entry, the single view and a second uninstall are 404, the catalog drops the skill and the marketplace entry reads `installed: false` again. +- `F25.long-description`: a skill description over 1024 characters is cut to 1024 in the project catalog, the installed view and the profile catalog, with a notice that names `invoke_skill(name=...)` and not the `SKILL.md` path. ## How to get to it (agent POV) @@ -860,6 +861,34 @@ Preconditions: the marketplace entry reads `installed: false` again, and the user tier no longer has `qa-f25-skill`. `qa-f25-src-skill` and `qa-f25-gone` stay installed in this run. +- **Long description (`F25.long-description`).** A project skill whose + description is longer than 1024 characters, read through the project + catalog, then installed and read through the installed view and the + profile launch catalog, then uninstalled. The bullet arranges everything + itself, so it also runs alone (`map run --only`). + ```sh + LD="$AGENT_SERVER_VERIFY_RUN/fixtures/qa-f25-long" + mkdir -p "$LD/.agents/skills/qa-f25-long" + printf -- '---\nname: qa-f25-long\ndescription: QA_LONG_DESC %s\n---\nQA_LONG_BODY\n' "$(printf 'x%.0s' $(seq 1500))" \ + > "$LD/.agents/skills/qa-f25-long/SKILL.md" + control-agent-server api POST /api/skills --json '{"load_public": false, "load_user": false, "load_project": true, "load_org": false, "project_dir": "'"$LD"'"}' \ + --expect 200 --check skills contains '"name": "qa-f25-long"' --check skills contains 'to load the full skill.' \ + --check skills not-contains 'complete skill content at' --save F25.long-description/project + control-agent-server api POST /api/skills/install --json '{"source": "'"$LD/.agents/skills/qa-f25-long"'"}' --expect 200 \ + --check name eq qa-f25-long --check description len-eq 1024 + control-agent-server api GET /api/skills/installed/qa-f25-long --expect 200 --check description len-eq 1024 \ + --check description contains 'QA_LONG_DESC x' --check description contains 'Call invoke_skill(name="qa-f25-long") to load the full skill.' \ + --check description not-contains SKILL.md --save F25.long-description/installed + control-agent-server api GET /api/skills/catalog --expect 200 --check skills contains '"name": "qa-f25-long"' \ + --check skills contains 'to load the full skill.' --check skills not-contains 'complete skill content at' --save F25.long-description/catalog + control-agent-server api DELETE /api/skills/installed/qa-f25-long --expect 200 + ``` + Every view cuts the description to 1024 characters when the `SKILL.md` + is loaded: it keeps the head and the tail and puts in the middle + `Due to the max output limit, only part of the + full description is shown. Call invoke_skill(name="qa-f25-long") to load + the full skill.`, naming the skill and never its file (the + agent's prompt omits skill locations on purpose). ## Gotchas diff --git a/.agents/skills/verify-agent-server/references/feature-map/F26-plugins.md b/.agents/skills/verify-agent-server/references/feature-map/F26-plugins.md index cb6d22807fe..228155b51c4 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F26-plugins.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F26-plugins.md @@ -43,7 +43,7 @@ Routes: `POST /api/plugins`, `POST /api/plugins/fetch`, `POST /api/plugins/insta - `F26.uninstall`: uninstall removes the directory and the record; the single view, the list and the ambient catalog no longer show it, and a second uninstall is 404. - `F26.conversation-ambient`: a new conversation loads an enabled installed plugin on its first message: its `UserPromptSubmit` hook runs (a `HookExecutionEvent`, also on the events socket), its command skill activates, and its skills show in the conversation. - `F26.conversation-disabled`: a conversation started after the plugin was disabled loads neither its hook nor its skills, while the user-directory plugins still load. -- `F26.conversation-resume-disabled`: after a restart, an existing conversation drops a since-disabled plugin entirely, its command as well as its hook. +- `F26.conversation-resume-disabled`: after a restart, an existing conversation drops a since-disabled plugin entirely: its command, skills and hook, and the MCP server its `.mcp.json` added. - `F26.restart-persist`: installed plugins, their records and their enabled flags survive a restart. - `F26.marketplace-local`: against a local stand-in for the extensions repository, `GET /api/plugins/marketplace` lists only `./plugins/` entries (local contents for local entries, attach coordinates for git entries) with a fresh `installed` flag; a repository without a manifest gives an empty catalog that is not cached. - `F26.marketplace-cache`: a non-empty catalog is served from a per-process cache (a new upstream entry appears only after a restart; the 5-minute TTL itself is not waited out), while `installed` stays fresh. @@ -672,9 +672,18 @@ Preconditions: ```sh control-agent-server api POST /api/plugins/install --json '{"source": "'"$PL"'", "force": true}' --expect 200 control-agent-server api PATCH /api/plugins/installed/qa-f26-inst --json '{"enabled": true}' --expect 200 + PMC=$(control-agent-server fixture plugin --name qa-f26-mcpplug --print-path) + MCPR=$(control-agent-server fixture mcp-server --name qa-f26-mcp | jq -c .mcp_server) + printf '{"mcpServers": {"qa-f26-mcp": %s}}\n' "$MCPR" > "$PMC/.mcp.json" + control-agent-server api POST /api/plugins/install --json '{"source": "'"$PMC"'", "force": true}' --expect 200 --check enabled eq true CIDR=$(control-agent-server conversation start --tools none --no-autotitle --prompt 'Reply with the single word: ok' --wait --timeout 240 --print-id) control-agent-server conversation events "$CIDR" --kinds HookExecutionEvent --contains QA_F26_INST_HOOK --expect-count 1 + control-agent-server api GET "/api/conversations/$CIDR" --query include_skills=true --expect 200 --check agent.mcp_config.qa-f26-mcp.command exists \ + --check agent.agent_context.skills contains '"name": "qa-f26-mcpplug-skill"' --quiet --save F26.conversation-resume-disabled/before + control-agent-server state cat base_state.json --conversation "$CIDR" --check ambient_plugin_mcp_servers contains qa-f26-mcp \ + --check ambient_plugin_skills contains qa-f26-inst:qa-hello control-agent-server api PATCH /api/plugins/installed/qa-f26-inst --json '{"enabled": false}' --expect 200 + control-agent-server api PATCH /api/plugins/installed/qa-f26-mcpplug --json '{"enabled": false}' --expect 200 control-agent-server restart control-agent-server ws start "/sockets/events/$CIDR" --name qa-f26-resume --duration 300 control-agent-server conversation send "$CIDR" --text '/qa-f26-inst:qa-hello' --wait --timeout 240 @@ -684,7 +693,11 @@ Preconditions: control-agent-server ws stop qa-f26-resume --kinds MessageEvent --contains '"activated_skills": ["qa-f26-inst:qa-hello"]' --expect-none \ --save F26.conversation-resume-disabled/frames control-agent-server api GET "/api/conversations/$CIDR" --query include_skills=true --expect 200 \ - --check agent.agent_context.skills not-contains qa-f26-inst --save F26.conversation-resume-disabled/conversation + --check agent.agent_context.skills not-contains qa-f26-inst --check agent.agent_context.skills not-contains qa-f26-mcpplug \ + --check agent.mcp_config.qa-f26-mcp missing --save F26.conversation-resume-disabled/conversation + control-agent-server state cat base_state.json --conversation "$CIDR" --check ambient_plugin_mcp_servers len-eq 0 \ + --check ambient_plugin_skills not-contains qa-f26-inst + control-agent-server api DELETE /api/plugins/installed/qa-f26-mcpplug --expect 200 ``` The conversation loaded the plugin before it was disabled (its hook ran for the prompt). Ambient plugins are not pinned to a conversation; on @@ -697,8 +710,16 @@ Preconditions: into the agent persisted in `base_state.json`, so the conversation records their skill and MCP server names and drops them after successful discovery. A failed scan preserves the previous contributions and ownership record. - Read from source and not driven here: MCP servers follow the same path; - explicit replacement agents start with no inherited ownership, and legacy + A second plugin, `qa-f26-mcpplug`, adds the stdio MCP server `qa-f26-mcp` + through its `.mcp.json`: before the restart the conversation's + `agent.mcp_config` has it and `base_state.json` records it in + `ambient_plugin_mcp_servers` (with the skills in `ambient_plugin_skills`; + neither field is in the REST view); after the plugin is disabled and the + conversation resumes, `agent.mcp_config` no longer has it and both records + hold only what is still enabled. The drop happens when the conversation + loads its plugins on the first resumed message, which is why the recipe + sends one before it reads the agent. That plugin is uninstalled at the end. + Read from source and not driven here: explicit replacement agents start with no inherited ownership, and legacy conversations without a record retain unidentifiable contributions rather than deleting caller content by name. The command is sent here for the first time because a keyword skill activates once per conversation diff --git a/.agents/skills/verify-agent-server/references/feature-map/F27-hooks-subagents-tools.md b/.agents/skills/verify-agent-server/references/feature-map/F27-hooks-subagents-tools.md index 048cc8102a5..fa88b77259f 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F27-hooks-subagents-tools.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F27-hooks-subagents-tools.md @@ -18,7 +18,7 @@ definitions (`agent_definitions`), and custom Python tool modules (`tool_module_qualnames`, `OH_EXTRA_PYTHON_PATH`, `--import-modules`), and the internal `start_child_conversation` tool, when a host attaches it. -Source: `openhands-agent-server/openhands/agent_server/tool_router.py`, `openhands-agent-server/openhands/agent_server/hooks_router.py`, `openhands-agent-server/openhands/agent_server/hooks_service.py`, `openhands-agent-server/openhands/agent_server/sub_agents_router.py`, `openhands-agent-server/openhands/agent_server/launch.py`, `openhands-agent-server/openhands/agent_server/__main__.py`, `openhands-sdk/openhands/sdk/tool/registry.py`, `openhands-sdk/openhands/sdk/tool/client_tool.py`, `openhands-sdk/openhands/sdk/hooks/`, `openhands-sdk/openhands/sdk/subagent/`, `openhands-tools/openhands/tools/preset/`, `openhands-tools/openhands/tools/task/`, `openhands-tools/openhands/tools/child_conversation/`, `openhands-tools/openhands/tools/browser_use/`, `clients/typescript/src/client/tool-client.ts`, `clients/typescript/src/client/hooks-client.ts`, `clients/typescript/src/client/sub-agents-client.ts` +Source: `openhands-agent-server/openhands/agent_server/tool_router.py`, `openhands-agent-server/openhands/agent_server/hooks_router.py`, `openhands-agent-server/openhands/agent_server/hooks_service.py`, `openhands-agent-server/openhands/agent_server/sub_agents_router.py`, `openhands-agent-server/openhands/agent_server/launch.py`, `openhands-agent-server/openhands/agent_server/__main__.py`, `openhands-sdk/openhands/sdk/tool/registry.py`, `openhands-sdk/openhands/sdk/tool/schema.py`, `openhands-sdk/openhands/sdk/tool/client_tool.py`, `openhands-sdk/openhands/sdk/hooks/`, `openhands-sdk/openhands/sdk/subagent/`, `openhands-tools/openhands/tools/preset/`, `openhands-tools/openhands/tools/task/`, `openhands-tools/openhands/tools/child_conversation/`, `openhands-tools/openhands/tools/browser_use/`, `clients/typescript/src/client/tool-client.ts`, `clients/typescript/src/client/hooks-client.ts`, `clients/typescript/src/client/sub-agents-client.ts` Needs: `llm`, `git`, `tmux`, `chromium` @@ -50,8 +50,9 @@ Routes: `GET /api/tools/`, `GET /api/tools/catalog`, `POST /api/sub-agents`, - `F27.client-tool-call`: when the agent calls a client tool, the events socket delivers a `ClientAction_` `ActionEvent` with the model's arguments, the server answers it with the `Tool call dispatched to client.` acknowledgment and the run finishes; the tool is registered again when the conversation resumes after a restart; a duplicate name in one request, a name that collides with `terminal` and a reused name with a different schema are each 422 and create nothing, while the same name with the same schema is accepted. - `F27.agent-definitions`: `agent_definitions` in the create request make that sub-agent available to the conversation's task tool, which delegates to it by name (the delegate runs the command, the `TaskObservation` names it); the definition is stored with the conversation and is not listed by `POST /api/sub-agents`. - `F27.custom-tool-modules`: with `OH_EXTRA_PYTHON_PATH` naming a directory, `tool_module_qualnames` imports a module from it at create, the agent runs its tool, the conversation reports the mapping and `GET /api/tools/` lists the tool (the sealed catalog does not); after a restart the tool is gone until the conversation is loaded again, and a new message runs it again; an unimportable module is a logged warning and the create still succeeds. +- `F27.tool-schema-keywords`: the tool `parameters` a custom tool module's `Action` produces in the model request keep its `pattern`, `const` and `additionalProperties: false`; today they are dropped (#6766). - `F27.import-modules`: a server started with `--import-modules` (and `OH_EXTRA_PYTHON_PATH`) lists the module's tool in `GET /api/tools/` and offers it in the catalog before any conversation exists; a module that cannot be imported stops the server before it binds its port. -- `F27.custom-tool-missing-on-resume`: when a conversation's tool module can no longer be imported after a restart, the server should drop the tool with its warning and keep the conversation readable; today its event search and a new message are 500 (`Unknown kind`). +- `F27.custom-tool-missing-on-resume`: when a conversation's tool module can no longer be imported after a restart, the server drops the tool with its warning, keeps the conversation readable, and answers a new message without a 500; today the message is 500 (`ToolDefinition ... is not registered`). - `F27.browser-playwright-path`: the browser probe finds a Playwright Chromium in `$PLAYWRIGHT_BROWSERS_PATH` as well as in standard paths, `~/.cache/ms-playwright` and `PATH`, and reports `browser_tool_set` usable. - `F27.browser-probe`: without a Chromium the server can find, the catalog reports `browser_tool_set` unusable and `/server_info` `usable_tools` leaves it out; with a Playwright Chromium in the server's `~/.cache/ms-playwright` and a restart, the catalog reports it usable and `usable_tools` lists it. - `F27.browser-disabled-launch`: a conversation started from the saved settings (default tools) gets `browser_tool_set` while the server finds Chromium, and leaves it out after a restart with `enable_browser: false`; a conversation created before keeps its stored tools. @@ -729,6 +730,65 @@ Preconditions: `F33.custom-tool-module-unimportable`). After a restart the registry no longer has the tool; loading the conversation for the next message imports the module again and the tool runs. +- **Tool schema keywords (`F27.tool-schema-keywords`), known bug #6766.** + A second custom tool module whose `Action` declares a `pattern`, a + one-value `Literal` and `extra="forbid"`; a conversation on an `llm-stub` + (it answers `finish`, so no real model is needed) offers the tool, and + the stub records the request's `tools`. + + Requires: `F27.custom-tool-modules` + ```sh + cat > "$F/pytools/qa_f27_strict_tools.py" <<'EOF' + from typing import Literal + + from pydantic import ConfigDict, Field + + from openhands.sdk import Action, Observation, ToolDefinition + from openhands.sdk.tool import ToolExecutor, register_tool + + + class QaF27StrictAction(Action): + model_config = ConfigDict(extra="forbid") + order_id: str = Field(pattern=r"^QAORD-[0-9]{6}$", description="Order id.") + currency: Literal["USD"] = Field(description="Currency.") + + + class QaF27StrictObservation(Observation): + pass + + + class QaF27StrictExecutor(ToolExecutor): + def __call__(self, action, conversation=None): + return QaF27StrictObservation.from_text(text=f"QA_F27_STRICT_RAN {action.order_id}") + + + class QaF27StrictTool(ToolDefinition[QaF27StrictAction, QaF27StrictObservation]): + @classmethod + def create(cls, conv_state=None, **params): + return [cls(description="QA strict tool.", action_type=QaF27StrictAction, + observation_type=QaF27StrictObservation, executor=QaF27StrictExecutor())] + + + register_tool("qa_f27_strict", QaF27StrictTool) + EOF + SST=$(control-agent-server fixture llm-stub --name qa-f27-schema --step 'tool:finish:{"message":"QA_F27_SCHEMA"}' --print-path) + SAG="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key\", \"base_url\": \"$SST/v1\", \"num_retries\": 0}, \"tools\": [{\"name\": \"qa_f27_strict\"}]}" + SCHEMA_CID=$(control-agent-server conversation start --workspace "$MW" --no-autotitle \ + --body-json "{\"agent\": $SAG, \"tool_module_qualnames\": {\"qa_f27_strict\": \"qa_f27_strict_tools\"}}" \ + --prompt 'qa-f27-schema' --wait --timeout 120 --print-id) + control-agent-server api GET "/api/conversations/$SCHEMA_CID" --expect 200 --check execution_status eq finished --quiet + control-agent-server sink read --name qa-f27-schema --contains '"name": "qa_f27_strict"' --expect-min 1 --save F27.tool-schema-keywords/request + control-agent-server sink read --name qa-f27-schema --contains 'QAORD-[0-9]{6}' --expect-min 1 # bug: ok eq false && requests eq 0 + ``` + The model request offers `qa_f27_strict` (the first read). Expected: its + `parameters` keep the model's constraints (`"pattern": + "^QAORD-[0-9]{6}$"` on `order_id`, `const`/one-value `enum` on + `currency`, top-level `additionalProperties: false`). The marked check + reads the pattern; when it passes, look for the other two in the saved + request before the marker goes. Today `Schema.to_mcp_schema()` keeps only + `type`, `description`, `properties`, `required`, `items` and `enum`, so + none of the three reaches the provider (#6766). The agent still validates a + call against the `Action` model locally. - **Startup imports (`F27.import-modules`).** `--import-modules` is a server argument passed by the owned shell-and-attach workflow: the bullet starts this checkout's server in plain shell with a private `HOME`, state @@ -740,7 +800,7 @@ Preconditions: mkdir -p "$IMP/home" "$F/import-missing/home" IPORT=$(python3 -c 'import socket; s = socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1])') (cd "$IMP" && exec env -i PATH="$PATH" HOME="$IMP/home" OH_PERSISTENCE_DIR="$IMP/persist" TMUX_TMPDIR="$IMP/tmux" \ - OH_SESSION_API_KEYS_0=qa-f27-import-key OH_EXTRA_PYTHON_PATH="$F/pytools" NO_PROXY=127.0.0.1,localhost COLUMNS=400 \ + OH_SESSION_API_KEYS_0=qa-f27-import-key OH_EXTRA_PYTHON_PATH="$F/pytools" NO_PROXY=127.0.0.1,localhost COLUMNS=400 LITELLM_LOCAL_MODEL_COST_MAP=True \ timeout 300 "$QA_PY" -m openhands.agent_server --host 127.0.0.1 --port "$IPORT" --import-modules qa_f27_tools) > "$IMP/server.log" 2>&1 & IPID=$! IRUN="" @@ -760,7 +820,7 @@ Preconditions: wait "$IPID" || true MPORT=$(python3 -c 'import socket; s = socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1])') if (cd "$F/import-missing" && exec env -i PATH="$PATH" HOME="$F/import-missing/home" OH_PERSISTENCE_DIR="$F/import-missing/persist" \ - OH_SESSION_API_KEYS_0=qa-f27-import-key OH_EXTRA_PYTHON_PATH="$F/pytools" NO_PROXY=127.0.0.1,localhost COLUMNS=400 \ + OH_SESSION_API_KEYS_0=qa-f27-import-key OH_EXTRA_PYTHON_PATH="$F/pytools" NO_PROXY=127.0.0.1,localhost COLUMNS=400 LITELLM_LOCAL_MODEL_COST_MAP=True \ timeout 120 "$QA_PY" -m openhands.agent_server --host 127.0.0.1 --port "$MPORT" --import-modules qa_f27_missing_module) > "$F/import-missing/server.log" 2>&1; then false; fi grep -q "ModuleNotFoundError: No module named 'qa_f27_missing_module'" "$F/import-missing/server.log" if grep -q 'Uvicorn running' "$F/import-missing/server.log"; then false; fi @@ -774,27 +834,34 @@ Preconditions: non-zero with the `ModuleNotFoundError` before uvicorn starts, so nothing is served and no state is written. `api --until-ok` gives up on a refused connection, hence the `curl --retry-connrefused` wait. -- **Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug.** + Both `env -i` launches set `LITELLM_LOCAL_MODEL_COST_MAP=True`: they + drop the proxy variables, so litellm's remote cost-map fetch fails, and its + background retry can crash the import with a circular import of + `litellm.rust_bridge.catalog` before the module is ever imported + (#6749). +- **Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug #6746.** Restart the run without `OH_EXTRA_PYTHON_PATH` and use the conversation of `F27.custom-tool-modules`, whose events include `qa_f27_echo` calls. Requires: `F27.custom-tool-modules` ```sh - W0=$(control-agent-server logs --grep 'Failed to import module' | jq -r .total_matching) + W0=$(control-agent-server logs --grep 'Failed to import' | jq -r .total_matching) control-agent-server restart --reset-config control-agent-server api GET "/api/conversations/$MCID" --expect 200 --check tool_module_qualnames.qa_f27_echo eq qa_f27_tools --quiet - control-agent-server api POST "/api/conversations/$MCID/events" --json '{"role": "user", "content": [{"type": "text", "text": "Reply with the single word ok."}], "run": false}' --expect 2xx,4xx --save F27.custom-tool-missing-on-resume/send # bug: status eq 500 && response.body.exception contains "Unknown kind 'QaF27EchoObservation'" - control-agent-server logs --grep 'Failed to import module' --expect-min "$((W0 + 1))" - control-agent-server api GET "/api/conversations/$MCID/events/search" --expect 200 --check items len-ge 1 --quiet # bug: status eq 500 + control-agent-server api POST "/api/conversations/$MCID/events" --json '{"role": "user", "content": [{"type": "text", "text": "Reply with the single word ok."}], "run": false}' --expect 2xx,4xx --save F27.custom-tool-missing-on-resume/send # bug: status eq 500 && response.body.exception contains "ToolDefinition 'qa_f27_echo' is not registered" + control-agent-server logs --grep 'Failed to import' --expect-min "$((W0 + 1))" + control-agent-server api GET "/api/conversations/$MCID/events/search" --expect 200 --check items len-ge 1 --quiet ``` `_prepare_persisted_runtime` logs `Failed to import module 'qa_f27_tools' ... Tool will not be available.` and promises to go on without the tool. - Today the conversation cannot be loaded at all: its stored events hold a - `QaF27EchoObservation`, whose class only that module defines, so the - message `POST` and `GET .../events/search` are `500` with `Unknown kind - 'QaF27EchoObservation'`. Only `GET /api/conversations/{id}`, which reads - the metadata without loading, still answers. A deployment that drops or - renames a custom tool module loses every conversation that used it. + The conversation still loads: `GET /api/conversations/{id}` answers and + `GET .../events/search` lists its stored events. Expected: a message is accepted, or refused with a + client error naming the missing tool. Today every message `POST` is an + unhandled `500` whose `exception` is `"ToolDefinition 'qa_f27_echo' is not + registered"`: the agent is built from the stored tool list on the first + message, and `resolve_tool` raises. A deployment that drops or renames a + custom tool module can read but no longer continue every conversation + that used it (#6746). - **Playwright browsers directory (`F27.browser-playwright-path`), blocked on a machine with Chromium at a standard path (needs Chromium only in a Playwright directory).** Point the server at the run-owned Playwright directory through `PLAYWRIGHT_BROWSERS_PATH` only. @@ -990,10 +1057,10 @@ Preconditions: a run. A model may retry a blocked call; the recipes allow several blocked attempts (`--expect-min`). - `tool_module_qualnames` changes process-wide registries, and a custom - tool's events need its module to deserialize + tool needs its module to continue a conversation (`F27.custom-tool-missing-on-resume`). The server log wraps long lines, so the recipes grep for short phrases - (`Extended sys.path with 1 directory`, `Failed to import module`). + (`Extended sys.path with 1 directory`, `Failed to import`). - `OH_EXTRA_PYTHON_PATH` only extends `sys.path`; nothing is imported until a create names the module or `--import-modules` does at startup. A launched run gets the variable with `restart --env`, but `--import-modules` diff --git a/.agents/skills/verify-agent-server/references/feature-map/F31-persistence-and-limits.md b/.agents/skills/verify-agent-server/references/feature-map/F31-persistence-and-limits.md index c350d6f3e17..0b2b37567eb 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F31-persistence-and-limits.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F31-persistence-and-limits.md @@ -24,7 +24,7 @@ Routes: none - `F31.lease-renewed`: while the server lives, the lease's `expires_at` moves forward within 15 s, with the same owner and generation. - `F31.lease-released-graceful`: a graceful restart deletes the lease of every loaded conversation, catalog reads do not claim it again, and the next load claims generation 1 under the new pid. - `F31.lease-dead-owner-takeover`: after a SIGKILL the lease still names the dead pid; the restarted server takes it over on first load with generation 2 and logs the takeover. -- `F31.crash-tool-call`: a conversation SIGKILLed during a tool call is loaded again at startup, comes back `error` with one `AgentErrorEvent` answering the interrupted call and no observation, and the resumed run hands that error to the model. +- `F31.crash-tool-call`: a conversation SIGKILLed during a tool call is loaded again at startup, comes back `error` with one `AgentErrorEvent` (`recovery_reason: server_restart`) answering the interrupted call and no observation, and the resumed run hands that error to the model. - `F31.state-survives-restarts`: a conversation's title, tags and events, the settings, a stored secret and the LLM profiles read back unchanged after a graceful restart and after a SIGKILL. - `F31.no-plaintext-at-rest`: after a real model run, the provider key, a stored secret and a conversation secret appear nowhere in the run's persisted files or the server log, while the server still hands out the provider key and the stored secret and keeps the conversation secret encrypted in its registry. - `F31.second-instance-catalog`: a second server sharing the conversations directory lists, counts and reads conversations that existed when it started, without claiming their leases. @@ -203,7 +203,8 @@ Preconditions: control-agent-server api GET "/api/conversations/$T" --check execution_status eq error --quiet --save F31.crash-tool-call/get control-agent-server api GET "/api/conversations/$T/events/search" --query kind=$AGENT_ERROR_KIND --check items len-eq 1 \ --check items.0.tool_call_id eq "$TCALL" --check items.0.error contains 'A restart occurred while this tool was in progress' \ - --save F31.crash-tool-call/agent-error + --check items.0.recovery_reason eq server_restart --check items.0.classification.kind eq internal \ + --check items.0.classification.retryable eq false --save F31.crash-tool-call/agent-error control-agent-server api GET "/api/conversations/$T/events/count" --query kind=$OBSERVATION_KIND --check . eq 0 control-agent-server fixture llm-stub --name qa-f31-tool --step 'tool:finish:{"message":"QA_F31_RESUMED"}' control-agent-server api POST "/api/conversations/$T/run" --expect 200 --check success eq true @@ -216,7 +217,9 @@ Preconditions: startup loaded `T` (its lease already names the new pid), marked it `error` and appended one `AgentErrorEvent` for the open `tool_call_id` ("A restart occurred while this tool was in progress ..."), with no - observation. The resumed model request carries that error as the tool + observation. The error says where it came from: `recovery_reason: + "server_restart"` (`null` on errors not synthesized by recovery) + and `classification` `{"kind": "internal", "retryable": false}`. The resumed model request carries that error as the tool result, and `T` finishes. - **State survives restarts (`F31.state-survives-restarts`).** One conversation, a settings field, a stored secret and the two DeepSeek diff --git a/.agents/skills/verify-agent-server/references/feature-map/F34-typescript-client.md b/.agents/skills/verify-agent-server/references/feature-map/F34-typescript-client.md index cb24b6bc75c..7c76cbd4ad5 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F34-typescript-client.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F34-typescript-client.md @@ -49,6 +49,7 @@ Routes: none - `F34.manager`: `ConversationManager` searches with paging, counts by status, batch-gets (unknown ids are `null`), lists all, loads a conversation with its server working directory, creates and deletes one, and an unknown id rejects with 404. - `F34.manager-update-result`: `ConversationManager.updateConversation` stores the new title and resolves with the updated `ConversationInfo` it is typed to return. - `F34.manager-create-options`: `ConversationManager.createConversation(agent, {maxIterations, stuckDetection})` creates the conversation with those settings. +- `F34.manager-batch-over-limit`: `getConversations` on `ConversationManager`, `ConversationClient` and the ACP namespace splits a list of more than 99 ids into requests the server accepts (it refuses 100 with 422) and returns every entry in input order, `null` for unknown ids. - `F34.manager-acp`: `ConversationManager.acp` creates a conversation for an inline ACP agent (with its `maxIterations`) and gets, batch-gets, lists and counts it with its ACP agent fields. - `F34.hooks-client`: `HooksClient.loadHooks` and `RemoteConversation.loadHooks` read a project's `.openhands/hooks.json` as a snake_case `HookConfig` the client's hook helpers understand, and `null` for a directory without hooks. - `F34.mcp-client`: `MCPClient.testServer` lists a stdio MCP server's tools and runs one tool call, reports a server that fails to start as `{ok: false}` with an `error_kind`, and an empty command rejects with 422. @@ -969,6 +970,44 @@ Preconditions: input order with `null` for the unknown id, the loaded conversation carries its server working directory, and the created conversation is gone after the delete (404, count back to `COUNT`). +- **Batches above the server limit (`F34.manager-batch-over-limit`).** + `GET /api/conversations?ids=` takes at most 99 ids; the client methods + split a longer list into requests of at most 99 and join the answers. + `CID`, then 150 unknown ids, then `SCID` go through + `ConversationManager.getConversations`, `ConversationClient.getConversations` + and the ACP namespace's `getConversations`. + + Requires: `F34.conversation-create` + ```sh + Q99=$(for N in $(seq 1 99); do printf 'ids=00000000-0000-4000-8000-%012d&' "$N"; done) + control-agent-server api GET "/api/conversations?${Q99%&}" --expect 200 --check . len-eq 99 --check 0 eq null --quiet + control-agent-server api GET "/api/conversations?${Q99}ids=00000000-0000-4000-8000-000000000100" --expect 422 \ + --check detail.0.type eq too_long --quiet --save F34.manager-batch-over-limit/rest-100 + cat > "$F34/batch.mjs" <<'JS' + import { writeFileSync } from 'node:fs'; + import { m, c, host, apiKey, F34, assert } from './lib.mjs'; + const [cid, scid] = process.argv.slice(2); + const unknown = Array.from({ length: 150 }, (_, i) => `00000000-0000-4000-8000-${String(i + 1).padStart(12, '0')}`); + const ids = [cid, ...unknown, scid]; + const expected = [cid, ...unknown.map(() => null), scid]; + const manager = new m.ConversationManager({ host, apiKey }); + const viaManager = (await manager.getConversations(ids)).map((x) => x && x.id); + const viaClient = (await new c.ConversationClient({ host, apiKey }).getConversations(ids)).map((x) => x && x.id); + const viaAcp = (await manager.acp.getConversations(ids)).map((x) => x && x.id); + writeFileSync(`${F34}/batch.json`, JSON.stringify({ manager: viaManager.length, client: viaClient.length, acp: viaAcp.length })); + assert.deepEqual(viaManager, expected); + assert.deepEqual(viaClient, expected); + assert.deepEqual(viaAcp, expected); + console.log('QA_F34_BATCH_OK'); + JS + control-agent-server exec --timeout 60 --expect-output QA_F34_BATCH_OK --save F34.manager-batch-over-limit/program -- node "$F34/batch.mjs" "$CID" "$SCID" + control-agent-server assert --json "$F34/batch.json" --check manager eq 152 --check client eq 152 --check acp eq 152 + ``` + REST answers 99 ids (all unknown, all `null`) and refuses a 100th with + `422` (`too_long`). Each client method resolves 152 entries in input + order, `CID` first, `SCID` last and `null` for every unknown id (the client + sends at most 99 ids per request, read from source, which the REST limit + above makes necessary). - **Update result (`F34.manager-update-result`).** `PATCH /api/conversations/{id}` answers `Success` (`{"success": true}`, also in the client's generated contract), so `ConversationManager.updateConversation` @@ -1504,3 +1543,10 @@ Preconditions: changes. The raw `/openapi.json` of a running server is not the public contract (it includes `/v1` and internal schemas), so regenerating from it shows thousands of unrelated changes. +- Agent Canvas pins `@openhands/typescript-client` 1.62.0, which sends every id + in one lookup, so a conversation with more than 99 children still gets a 422 + there until its pin moves to a release with the batched lookup + (`F34.manager-batch-over-limit`). The release bot moves the agent-server and + client pins together after each SDK release, as + [OpenHands/OpenHands#19658](https://github.com/OpenHands/OpenHands/pull/19658) + did for 1.62.0. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F37-reactions.md b/.agents/skills/verify-agent-server/references/feature-map/F37-reactions.md index fac242646fe..82013965609 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F37-reactions.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F37-reactions.md @@ -2,9 +2,14 @@ A reaction is a small emoji on a message: the agent's way to say "seen, working on it" before it answers, and the user's way to acknowledge an answer without -writing back. Both are saved as a non-LLM `ReactionEvent` (`target_event_id`, -`emoji`, `removed`, `source`), so clients fold them into each message's current -set and the model never sees them. The agent reacts with the `react` and +writing back. Both are saved as a `ReactionEvent` (`target_event_id`, +`emoji`, `removed`, `source`; a user's reaction also keeps the end of the +message's text in `target_text_ending`, at most 500 characters, with +`target_text_clipped`), and clients fold them into each message's current set. +The model sees the user's reactions on its next turn, as one user line such +as `[The user reacted ❤️ to this message:]` followed by that text ending; its +own reactions stay out of what it sees, since its `react` call is already +there. The agent reacts with the `react` and `unreact` tools, whose executor only reads the log: the observation carries the reaction and the agent records the `ReactionEvent` right after it, on its own thread. The user reacts through `POST /api/conversations/{id}/reactions`, @@ -19,8 +24,12 @@ and new conversations do not get the tools. Flags are listed with `GET /api/settings/experimental` and switched with `PATCH /api/settings/experimental/{key}` `{"enabled": bool}`, stored in `PersistedSettings.experimental_features`, and read when a conversation starts. +A server inside a Docker runtime has no settings of its own: the host passes +the user's switches in `OH_RUNTIME_EXPERIMENTAL_FEATURES` (a JSON object), which +then picks the tools of new conversations, and that server's REST route stops +checking the switch because the host checks it before forwarding. -Source: `openhands-sdk/openhands/sdk/event/reaction.py`, `openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py` (`react`), `openhands-sdk/openhands/sdk/agent/agent.py` (`follow_up_events`), `openhands-tools/openhands/tools/reaction/`, `openhands-agent-server/openhands/agent_server/event_router.py`, `openhands-agent-server/openhands/agent_server/experimental_features.py`, `openhands-agent-server/openhands/agent_server/experimental_features_router.py`, `openhands-sdk/openhands/sdk/launch/finalize.py` (`experimental_feature_tools`) +Source: `openhands-sdk/openhands/sdk/event/reaction.py`, `openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py` (`react`), `openhands-sdk/openhands/sdk/agent/agent.py` (`follow_up_events`), `openhands-tools/openhands/tools/reaction/`, `openhands-agent-server/openhands/agent_server/event_router.py`, `openhands-agent-server/openhands/agent_server/experimental_features.py`, `openhands-agent-server/openhands/agent_server/experimental_features_router.py`, `openhands-sdk/openhands/sdk/launch/finalize.py` (`experimental_feature_tools`), `openhands-sdk/openhands/sdk/context/view/view.py`, `openhands-agent-server/openhands/agent_server/docker_runtime/routers.py`, `clients/typescript/src/client/conversation-client.ts`, `clients/typescript/src/client/settings-client.ts` Needs: `llm` @@ -29,19 +38,28 @@ Routes: `POST /api/conversations/{conversation_id}/reactions` ## Sub-features - `F37.flag`: `GET /api/settings/experimental` lists `conversation_reactions` with `enabled: false` on a fresh server; `PATCH .../conversation_reactions` `{"enabled": true}` returns it on and the list agrees; an unknown key is 404. -- `F37.flag-gates`: with the flag off, `POST .../reactions` is 409 and a new conversation has no `react`/`unreact`; with it on, a new conversation started without naming them has both tools. +- `F37.flag-gates`: with the flag off, `POST .../reactions` is 409 and a new conversation has no `react`/`unreact`; with it on, a new conversation started without naming them has both tools. Turning it off again refuses the route at once (409 `Conversation reactions are turned off`) while that conversation keeps its tools. - `F37.rest`: `POST .../reactions` with `{emoji, target_event_id}` saves a user `ReactionEvent` on that message and returns it; `removed: true` saves the removal; the conversation stays `idle`. - `F37.rest-validation`: an emoji outside the set is 422, an unknown target is 400, and leaving out the target with no agent message yet is 400; nothing is saved. - `F37.tool`: an agent with `react` reacts to the user's latest message when asked to react first, and leaves the reaction on after it answers: the log has the tool's `ObservationEvent` followed by one agent `ReactionEvent` whose target is that message, and no removal. - `F37.rest-while-confirming`: a reaction posted while a tool call waits for confirmation answers 200 but is held back: it joins the log right after the call's result once the call is accepted, never between the call and its result, and the run finishes. - `F37.rest-default`: after the agent has answered, a reaction without `target_event_id` goes on its answer: the latest agent message or `finish`. +- `F37.agent-sees-user-reactions`: a user's reaction on the agent's `finish` keeps the answer's text in `target_text_ending`, and the next model request carries it as `[The user reacted ❤️ to this message:]` plus that text, while the agent's own `react` reaction never appears as a reaction line. +- `F37.rest-active-branch`: after navigating back before the agent's answer, the abandoned answer is 400 `No message with id ...`, a reaction without a target is 400 `There is no agent message to react to.`, an event that is not a message is 400 `Event ... is not a message.`, and the user's message on the active branch still takes a reaction. +- `F37.ts-client`: the TypeScript `SettingsClient` lists and switches the flag (an unknown key rejects with a 404 `HttpError`) and `ConversationClient.react` adds and removes a reaction that a GET of the event shows; a reaction with nothing to target rejects with a 400 `HttpError`. +- `F37.runtime-env`: with `OH_RUNTIME_EXPERIMENTAL_FEATURES='{"conversation_reactions": true}'` and the stored flag off, new conversations get both tools and `POST .../reactions` answers 200; with `{}` new conversations get neither. +- `F37.rest-response-parent`: the `ReactionEvent` that `POST .../reactions` returns is the event as stored, with the same `parent_id` (the reacted message when it is the active leaf) that search, get-one and the socket show. ## How to get to it (agent POV) - REST: `POST /api/conversations/{conversation_id}/reactions` with - `{"emoji", "target_event_id"?, "removed"?}` returns the saved - `ReactionEvent`. Without `target_event_id` it targets the agent's latest - message or `finish` action. It never runs the agent. + `{"emoji", "target_event_id"?, "removed"?}` returns the `ReactionEvent` + as stored (a reaction held back while a tool call waits has no `parent_id` + until it joins the events). Without `target_event_id` it targets the + agent's latest message or `finish` action; any message on the active branch, + the user's own included, can be targeted. It never runs the agent. +- REST: `GET /api/settings/experimental` and + `PATCH /api/settings/experimental/{key}` (owned by F18) switch the flag. - Agent tools (registered by the agent-server, not part of the default preset): `react` and `unreact` from `openhands.tools.reaction`, with `{"emoji", "message_id"?}`; without `message_id` they target the user's @@ -50,6 +68,15 @@ Routes: `POST /api/conversations/{conversation_id}/reactions` - Events: `ReactionEvent` in search, get-one and the socket like other events; the latest event for a (message, emoji, source) is the current state. - SDK: `LocalConversation.react(emoji, target_event_id=None, removed=False)`. +- TypeScript client: `ConversationClient.react(conversationId, emoji, + {targetEventId?, removed?})`, `SettingsClient.listExperimentalFeatures()` + and `updateExperimentalFeature(key, enabled)`; the `ReactionEvent` type and + `isReactionEvent` guard (`clients/typescript/src/events/types.ts`). Recipe + `F37.ts-client` runs them on the built package (`clients/typescript/dist`). +- Configuration: `OH_RUNTIME_EXPERIMENTAL_FEATURES` (JSON object of switches, + set by a Docker-runtime host for its container's server) replaces the stored + switches for new conversations and turns off the route's own check + (`F37.runtime-env`). ## Driving it with control-agent-server @@ -76,10 +103,19 @@ Preconditions: --expect 200 --check enabled eq true control-agent-server api PATCH /api/settings/experimental/time_travel --json '{"enabled": true}' --expect 404 ON=$(control-agent-server conversation start --no-autotitle --tools none --print-id) - control-agent-server api GET "/api/conversations/$ON" --check agent.tools len-eq 2 --save F37.flag-gates/tools + control-agent-server api GET "/api/conversations/$ON" --check agent.tools len-eq 2 --check agent.tools.0.name eq react \ + --check agent.tools.1.name eq unreact --save F37.flag-gates/tools control-agent-server api GET "/api/conversations/$OFF" --check agent.tools len-eq 0 + control-agent-server api PATCH /api/settings/experimental/conversation_reactions --json '{"enabled": false}' --expect 200 + control-agent-server api POST "/api/conversations/$ON/reactions" --json '{"emoji": "eyes"}' --expect 409 \ + --check detail eq 'Conversation reactions are turned off' --save F37.flag-gates/off-again + control-agent-server api GET "/api/conversations/$ON" --check agent.tools len-eq 2 + control-agent-server api PATCH /api/settings/experimental/conversation_reactions --json '{"enabled": true}' --expect 200 \ + --check enabled eq true ``` - The flag stays on for the rest of this family. + Switching the flag off again refuses the route at once, while the + conversation started under it keeps `react` and `unreact`. The flag is on + again, and stays on for the rest of this family. - **React over REST (`F37.rest`).** Requires: `F37.flag` ```sh @@ -158,6 +194,146 @@ Preconditions: the user asked for and no condensation. A reaction between a call and its result would make the model's API refuse the history. Held reactions live in memory: a server restart while the call waits drops them. +- **What the model sees (`F37.agent-sees-user-reactions`).** The prompt + the server sends is observable only at the provider, so this bullet records + it with an `llm-stub` (a recorder here, not a stand-in for a happy path): the + stub reacts with `eyes`, then answers with `finish`; the user puts `heart` + on that answer; the next request carries the user's reaction line and no + line for the agent's own reaction. + Requires: `F37.flag` + ```sh + SEE=$(control-agent-server fixture llm-stub --name qa-f37-see --step 'tool:react:{"emoji":"eyes"}' \ + --step 'tool:finish:{"message":"QA_F37_ANSWER"}' --print-path) + SEE_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key\", \"base_url\": \"$SEE/v1\", \"num_retries\": 0}, \"tools\": []}" + SC=$(control-agent-server conversation start --no-autotitle --body-json "{\"agent\": $SEE_AGENT}" --prompt 'QA_F37_SEE first' \ + --wait --timeout 120 --print-id) + control-agent-server api GET "/api/conversations/$SC/events/search" --query kind=ReactionEvent \ + --check items len-eq 1 --check items.0.source eq agent --check items.0.emoji eq eyes + control-agent-server api POST "/api/conversations/$SC/reactions" --json '{"emoji": "heart"}' --expect 200 \ + --check source eq user --check target_text_ending eq QA_F37_ANSWER --check target_text_clipped eq false \ + --save F37.agent-sees-user-reactions/add + control-agent-server sink read --name qa-f37-see --expect-min 2 --expect-max 2 + control-agent-server conversation send "$SC" --text 'QA_F37_SEE second' --wait --timeout 120 + control-agent-server sink read --name qa-f37-see --contains 'Reacted :eyes:' --expect-min 2 + control-agent-server sink read --name qa-f37-see --contains 'The user reacted' --expect-min 1 --expect-max 1 \ + --save F37.agent-sees-user-reactions/next-request + control-agent-server sink read --name qa-f37-see --contains 'The user reacted \u2764\ufe0f to this message:]\nQA_F37_ANSWER' \ + --expect-min 1 --expect-max 1 + control-agent-server sink read --name qa-f37-see --contains 'You reacted' --expect-max 0 + ``` + Only the third request (after the user's reaction) holds the reaction line, + with the answer's text after it (the sink keeps the request as JSON, so the + glyph and the line break read as `\u2764\ufe0f` and `\n`). The agent's `react` call and its + `Reacted :eyes:.` result are in every later request, but its + `ReactionEvent` never becomes a line of its own. +- **Targets come from the active branch (`F37.rest-active-branch`).** Move + the conversation back to the user's first message, before the answer. + Requires: `F37.agent-sees-user-reactions` + ```sh + U1=$(control-agent-server api GET "/api/conversations/$SC/events/search" --query kind=MessageEvent --query body=QA_F37_SEE \ + --query sort_order=TIMESTAMP --field items.0.id) + FIN=$(control-agent-server api GET "/api/conversations/$SC/events/search" --query kind=ActionEvent --query sort_order=TIMESTAMP \ + --query source=agent --field items.1.id) + OBS=$(control-agent-server api GET "/api/conversations/$SC/events/search" --query kind=ObservationEvent --field items.0.id) + control-agent-server api POST "/api/conversations/$SC/reactions" --json "{\"emoji\": \"eyes\", \"target_event_id\": \"$OBS\"}" \ + --expect 400 --check detail eq "Event $OBS is not a message." --save F37.rest-active-branch/not-a-message + control-agent-server api POST "/api/conversations/$SC/navigate" --json "{\"event_id\": \"$U1\"}" --expect 200 + control-agent-server api POST "/api/conversations/$SC/reactions" --json "{\"emoji\": \"eyes\", \"target_event_id\": \"$FIN\"}" \ + --expect 400 --check detail eq "No message with id $FIN." --save F37.rest-active-branch/abandoned + control-agent-server api POST "/api/conversations/$SC/reactions" --json '{"emoji": "eyes"}' --expect 400 \ + --check detail eq 'There is no agent message to react to.' + control-agent-server api POST "/api/conversations/$SC/reactions" --json "{\"emoji\": \"tada\", \"target_event_id\": \"$U1\"}" \ + --expect 200 --check target_event_id eq "$U1" --check target_text_ending eq 'QA_F37_SEE first' + ``` + The agent's `finish` is still stored but no longer on the active branch, so + it cannot be reacted to; the user's own message can. +- **TypeScript client (`F37.ts-client`).** A program drives + `SettingsClient` and `ConversationClient` against the run. + Needs: `node` + Requires: `F37.flag` + ```sh + test -f clients/typescript/dist/clients.js || (cd clients/typescript && npm ci && npm run build) + TSC=$(control-agent-server conversation start --no-autotitle --tools none --print-id) + cat > "$AGENT_SERVER_VERIFY_RUN/qa-f37-ts.mjs" <<'JS' + import assert from 'node:assert/strict'; + import { resolve } from 'node:path'; + import { pathToFileURL } from 'node:url'; + + const dist = (file) => pathToFileURL(resolve('clients/typescript/dist', file)).href; + const { SettingsClient, ConversationClient } = await import(dist('clients.js')); + const { isHttpError } = await import(dist('index.js')); + const opts = { host: process.env.AGENT_SERVER_URL, apiKey: process.env.SESSION_API_KEY }; + const settings = new SettingsClient(opts); + const conversations = new ConversationClient(opts); + const [cid, target, empty] = [process.env.QA_F37_CID, process.env.QA_F37_TARGET, process.env.QA_F37_EMPTY]; + + const flags = await settings.listExperimentalFeatures(); + assert.deepEqual(flags.map((f) => [f.key, f.enabled]), [['conversation_reactions', true]]); + assert.equal((await settings.updateExperimentalFeature('conversation_reactions', true)).enabled, true); + await assert.rejects(settings.updateExperimentalFeature('qa_f37_unknown', true), + (err) => isHttpError(err) && err.status === 404); + + const added = await conversations.react(cid, 'tada', { targetEventId: target }); + assert.equal(added.kind, 'ReactionEvent'); + assert.equal(added.source, 'user'); + assert.equal(added.removed, false); + const removed = await conversations.react(cid, 'tada', { targetEventId: target, removed: true }); + assert.equal(removed.removed, true); + await assert.rejects(conversations.react(empty, 'eyes'), + (err) => isHttpError(err) && err.status === 400 && err.detail === 'There is no agent message to react to.'); + console.log('QA_F37_TS_OK', added.id, removed.id); + JS + OUT=$(control-agent-server exec --env QA_F37_CID="$RC" --env QA_F37_TARGET="$RM" --env QA_F37_EMPTY="$TSC" --timeout 120 \ + --expect-output QA_F37_TS_OK --save F37.ts-client/program -- node "$AGENT_SERVER_VERIFY_RUN/qa-f37-ts.mjs") + TS_ADDED=$(printf '%s' "$OUT" | grep -o 'QA_F37_TS_OK [0-9a-f-]* [0-9a-f-]*' | cut -d' ' -f2) + test -n "$TS_ADDED" + control-agent-server api GET "/api/conversations/$RC/events/$TS_ADDED" --expect 200 --check kind eq ReactionEvent \ + --check emoji eq tada --check target_event_id eq "$RM" --save F37.ts-client/added + ``` + The flag reads on, the unknown key rejects with 404, and the reaction the + client added is in the conversation's log. +- **Inside a Docker runtime (`F37.runtime-env`).** Translated: the env + variable a Docker-runtime host passes to its container's server is set on + this run's server directly, with the stored flag off. + Requires: `F37.flag` + ```sh + control-agent-server api PATCH /api/settings/experimental/conversation_reactions --json '{"enabled": false}' --expect 200 + control-agent-server restart --env 'OH_RUNTIME_EXPERIMENTAL_FEATURES={"conversation_reactions": true}' + control-agent-server api GET /api/settings/experimental --check 0.enabled eq false + RT=$(control-agent-server conversation start --no-autotitle --tools none --print-id) + control-agent-server api GET "/api/conversations/$RT" --check agent.tools len-eq 2 --save F37.runtime-env/tools + control-agent-server api POST "/api/conversations/$RC/reactions" --json "{\"emoji\": \"+1\", \"target_event_id\": \"$RM\"}" \ + --expect 200 --check emoji eq '+1' --save F37.runtime-env/react + control-agent-server restart --reset-config --env 'OH_RUNTIME_EXPERIMENTAL_FEATURES={}' + RT0=$(control-agent-server conversation start --no-autotitle --tools none --print-id) + control-agent-server api GET "/api/conversations/$RT0" --check agent.tools len-eq 0 + control-agent-server restart --reset-config + control-agent-server api POST "/api/conversations/$RC/reactions" --json "{\"emoji\": \"+1\", \"target_event_id\": \"$RM\"}" --expect 409 + ``` + With the variable set, the server takes its switches from it and leaves + the route's check to the host; without it, the stored (off) flag applies + again. + +- **The response is the stored event (`F37.rest-response-parent`).** + A reaction on the latest message is stamped with that message as its + parent; the POST answer carries the same `parent_id` as the stored copy. + Requires: `F37.flag` + ```sh + control-agent-server api PATCH /api/settings/experimental/conversation_reactions --json '{"enabled": true}' --expect 200 + PC=$(control-agent-server conversation start --no-autotitle --tools none --print-id) + control-agent-server api POST "/api/conversations/$PC/events" \ + --json '{"role": "user", "content": [{"type": "text", "text": "QA_F37_PARENT"}], "run": false}' --expect 200 + PM=$(control-agent-server api GET "/api/conversations/$PC/events/search" --query body=QA_F37_PARENT --field items.0.id) + PR=$(control-agent-server api POST "/api/conversations/$PC/reactions" --json "{\"emoji\": \"heart\", \"target_event_id\": \"$PM\"}" \ + --expect 200 --field id) + control-agent-server api GET "/api/conversations/$PC/events/$PR" --check parent_id eq "$PM" --save F37.rest-response-parent/stored + control-agent-server api POST "/api/conversations/$PC/reactions" --json "{\"emoji\": \"+1\", \"target_event_id\": \"$PM\"}" \ + --expect 200 --check parent_id eq "$PR" --save F37.rest-response-parent/post + ``` + The first reaction is read back to show the stored `parent_id`; the second + is stamped with the first as its parent (the active leaf), and its POST + answer says so: `LocalConversation.react` returns the event as the log + stored it. ## Gotchas @@ -177,3 +353,7 @@ Preconditions: abandoned branch cannot be reacted to. - A client older than this event kind cannot parse a conversation's events once it holds a reaction, as with any new event kind. +- Agent Canvas does not call the reaction or experimental-feature routes yet + and pins a release without them; its Features page and reaction chips are + [OpenHands/OpenHands#19505](https://github.com/OpenHands/OpenHands/issues/19505) + (PR [#19503](https://github.com/OpenHands/OpenHands/pull/19503)). diff --git a/.agents/skills/verify-agent-server/references/feature-map/README.md b/.agents/skills/verify-agent-server/references/feature-map/README.md index 482b7cb7a78..747db7c048d 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/README.md +++ b/.agents/skills/verify-agent-server/references/feature-map/README.md @@ -8,9 +8,9 @@ consumers). Read this index before driving the server, then use the matching family file as the recipe. -Maintenance baseline: main@4db1ad742ad143ddc3411631f04efb70323f40f9 (2026-11-09). The first pass; the next maintenance pass starts from this commit, a pass proposes the next baseline in its PR, and merging that PR accepts it. +Maintenance baseline: main@fb75ae907585c1634a85fd5168a795dd48eb4d65 (2026-11-12). The next maintenance pass starts from this commit, a pass proposes the next baseline in its PR, and merging that PR accepts it. -The map has **962 sub-features**. +The map has **990 sub-features**. ## Baseline preconditions @@ -69,17 +69,17 @@ them honest: [../maintenance.md](../maintenance.md). | F02 | [Authentication and workspace sessions](F02-auth-and-sessions.md) | session keys on HTTP and on WebSockets (first frame, deprecated query and header forms, close codes), several and rotated keys, no-auth and legacy modes, the default bind address, the workspace session cookie, CORS | `X-Session-API-Key`, `POST`/`DELETE /api/auth/workspace-session`, WebSocket auth on every socket | recipes: tmux | 21 | | F03 | [Deferred init (warm-pool dormant mode)](F03-deferred-init.md) | warm-pool dormant mode: the gate on `/api`, `/v1`, `/app-backends` and every socket (checked before auth), init key auth, validation and redaction, rollback, one-shot init, and every field the init body delivers (keys, secret key, paths, worktree root, run limit, CORS, webhooks, env, web URL) | `GET`/`POST /api/init`, `X-Init-API-Key` | git; launch `--no-auth --deferred-init` | 25 | | F04 | [Conversation lifecycle and catalog](F04-conversation-lifecycle.md) | create from settings, an agent profile or an inline agent; client ids, parents, worktrees, start-time secrets, 422 redaction; get, batch get, search, count; PATCH; final response; DELETE; restart | `POST`/`GET`/`PATCH`/`DELETE /api/conversations[/{id}]`, `/search`, `/count`, `/agent_final_response`, `/children` | llm, git | 36 | -| F05 | [Run, pause, interrupt and status transitions](F05-run-pause-interrupt.md) | run, pause and interrupt during model and tool calls; status transitions on every view; stuck detection and max_iterations stops; 409 and 429; graceful and SIGKILL restarts | `POST /api/conversations/{id}/run`, `/pause`, `/interrupt` | llm, tmux; launch `--config-json '{"max_concurrent_runs": 1}'` | 31 | -| F06 | [Conversation events and the events WebSocket](F06-conversation-events.md) | appending messages, event search, count, get and batch get with filters and paging, the events WebSocket (auth, snapshot, replay, live frames), history across restart | `POST`/`GET /api/conversations/{id}/events[...]`, `WS /sockets/events/{id}` | llm; launch `--env TZ=UTC` | 29 | -| F07 | [Session WebSocket protocol](F07-session-socket.md) | the resumable session socket: sync, durable and transient frames, seq replay, inbound messages, error and close codes, streaming deltas, backpressure, subscriber cap | `WS /sockets/session/{id}` | llm | 24 | +| F05 | [Run, pause, interrupt and status transitions](F05-run-pause-interrupt.md) | run, pause and interrupt during model and tool calls; status transitions on every view; stuck detection and max_iterations stops; 409 and 429; graceful and SIGKILL restarts | `POST /api/conversations/{id}/run`, `/pause`, `/interrupt` | llm, tmux; launch `--config-json '{"max_concurrent_runs": 1}'` | 32 | +| F06 | [Conversation events and the events WebSocket](F06-conversation-events.md) | appending messages, event search, count, get and batch get with filters and paging, the events WebSocket (auth, snapshot, replay, live frames), history across restart | `POST`/`GET /api/conversations/{id}/events[...]`, `WS /sockets/events/{id}` | llm; launch `--env TZ=UTC` | 30 | +| F07 | [Session WebSocket protocol](F07-session-socket.md) | the resumable session socket: sync, durable and transient frames, seq replay, inbound messages, error and close codes, streaming deltas, backpressure, subscriber cap | `WS /sockets/session/{id}` | llm | 26 | | F08 | [Confirmation policy, security analyzer and conversation secrets](F08-confirmation-security-secrets.md) | confirmation policies and accept/reject, the security analyzer, runtime conversation secrets (static, lookup, masking, encryption, no cipher), and what survives a restart | `POST /api/conversations/{id}/confirmation_policy`, `/security_analyzer`, `/secrets`, `/events/respond_to_confirmation` | llm, tmux | 23 | -| F09 | [Model, profile and ACP switching; runtime plugins](F09-model-switching-plugins.md) | swap the agent LLM, switch to a saved profile, switch an ACP model, load a plugin at runtime; stats, hooks and restart persistence | `POST /api/conversations/{id}/switch_llm`, `/switch_profile`, `/switch_acp_model`, `/load_plugin` | llm, tmux, node, network | 28 | +| F09 | [Model, profile and ACP switching; runtime plugins](F09-model-switching-plugins.md) | swap the agent LLM, switch to a saved profile, switch an ACP model, load a plugin at runtime; stats, hooks and restart persistence | `POST /api/conversations/{id}/switch_llm`, `/switch_profile`, `/switch_acp_model`, `/load_plugin` | llm, tmux, node, network | 37 | | F10 | [Fork, navigate, condense and ask_agent](F10-fork-navigate-condense-ask.md) | fork, navigate between branches, condense with a real summarizer, ask the agent a side question; state after restart | `POST /api/conversations/{id}/fork`, `/navigate`, `/condense`, `/ask_agent` | llm | 25 | | F11 | [Goal loop](F11-goals.md) | goal loops: start, LLM-judged completion and caps, stop, resume, supersede, interrupt, failures, busy and capacity limits | `POST /api/conversations/{id}/goal`, `/goal/stop`, `/goal/resume` | llm, tmux; launch `--config-json '{"max_concurrent_runs": 1}'` | 25 | | F12 | [Conversation runtime info, sandbox pause and credential bindings](F12-runtime-and-credentials.md) | runtime info and reprovision, prepare-for-sandbox-pause, credential bindings for ACP agents, managed LLM key refresh, the Docker runtime mode's routes and contract | `GET /api/conversations/{id}/runtime`, `POST .../runtime/reprovision`, `POST /api/conversations/prepare-for-sandbox-pause`, `PUT .../credential-bindings/{name}`, Docker mode `DELETE .../runtime`, `POST .../runtime/credentials` | llm, node, network | 31 | | F13 | [Bash commands and bash events](F13-bash.md) | global and conversation-scoped shell commands: execute, start, stop, timeouts, large output, events search and clear, retention, the bash events socket | `/api/bash/*`, `/api/conversations/{id}/bash/*`, `WS /sockets/bash-events` | baseline | 29 | | F14 | [File upload, download, archive and directory helpers](F14-files.md) | upload, download, create directory, home and subdirectory search, git-delta and tar.gz archives, trajectory zips, conversation-scoped variants and their path guard | `/api/file/*`, `/api/conversations/{id}/file/*` | llm, git, node | 30 | -| F15 | [Git changes, diffs, commits and repository search](F15-git.md) | working-tree changes and diffs, explicit refs, nested repositories, commit history and per-commit changes, scoped variants, GitHub repository search | `/api/git/*`, `/api/conversations/{id}/git/*`, `GET /api/git/repositories/search` | llm, git, node | 29 | +| F15 | [Git changes, diffs, commits and repository search](F15-git.md) | working-tree changes and diffs, explicit refs, nested repositories, commit history and per-commit changes, scoped variants, GitHub repository search | `/api/git/*`, `/api/conversations/{id}/git/*`, `GET /api/git/repositories/search` | llm, git, node | 36 | | F16 | [Workspace file serving and VS Code endpoints](F16-workspace-serving-and-vscode.md) | the per-conversation static workspace server (index, content types, ranges, traversal, cookie auth) and the VS Code status and URL routes | `GET /api/conversations/{id}/workspace[/{path}]`, `/api/vscode/*`, `/api/conversations/{id}/vscode/*` | llm, git | 26 | | F17 | [Workspace registry and parents](F17-workspaces-registry.md) | saved workspaces and parents: add, list, delete, validation, persistence, concurrent writers, corrupted files | `GET`/`POST`/`DELETE /api/workspaces`, `/api/workspaces/parents` | git, node | 24 | | F18 | [Server settings, settings schemas and the secrets store](F18-settings-and-secrets.md) | persisted settings with redacted, plaintext and encrypted exposure, sparse PATCH, settings schemas, the named secrets store, encryption at rest, cipher-key changes | `GET`/`PATCH /api/settings`, `/api/settings/agent-schema`, `/conversation-schema`, `/api/settings/secrets[/{name}]` | git, node | 32 | @@ -89,17 +89,17 @@ them honest: [../maintenance.md](../maintenance.md). | F22 | [Meta-profiles (model router)](F22-meta-profiles.md) | meta-profiles (model router): save, read, activate, delete, settings reflection, routing in a conversation | `/api/meta-profiles[/{name}]`, `/activate` | llm, node | 24 | | F23 | [LLM catalog and provider connections](F23-llm-catalog-and-connections.md) | provider and model catalogs, verified models, provider connections (create, rotate, delete, references from profiles), live ACP discovery and errors | `GET /api/llm/providers`, `/api/llm/models[/verified]`, `/api/llm/provider-connections[/{id}]`, `POST /api/acp/models` | llm, node, network | 28 | | F24 | [OpenAI ChatGPT subscription device flow](F24-openai-subscription.md) | the ChatGPT subscription device flow: models, status, device start and poll, logout, upstream failures | `/api/llm/subscription/openai/*` | node, network | 20 | -| F25 | [Skills catalog, install and marketplace](F25-skills.md) | the merged skill catalog, public-repo sync, installed skills (install, enable, refresh, uninstall), marketplace, trigger-word activation in a conversation | `POST /api/skills`, `/api/skills/sync`, `/api/skills/install`, `/api/skills/installed[/{name}]`, `/api/skills/marketplace` | git; recipes: llm, network, node | 34 | +| F25 | [Skills catalog, install and marketplace](F25-skills.md) | the merged skill catalog, public-repo sync, installed skills (install, enable, refresh, uninstall), marketplace, trigger-word activation in a conversation | `POST /api/skills`, `/api/skills/sync`, `/api/skills/install`, `/api/skills/installed[/{name}]`, `/api/skills/marketplace` | git; recipes: llm, network, node | 35 | | F26 | [Plugins catalog, install and marketplace](F26-plugins.md) | plugin catalog, install from a path or git source, enable, refresh, uninstall, marketplace, loading a plugin into a conversation | `POST /api/plugins`, `/api/plugins/install`, `/api/plugins/installed[/{name}]`, `/api/plugins/marketplace`, `/api/plugins/fetch` | llm, git, node, network | 26 | -| F27 | [Hooks, sub-agents and tool catalogs](F27-hooks-subagents-tools.md) | hooks and sub-agent catalogs, blocking hooks in a conversation, the tool registry and catalog, client tools, forwarded agent definitions, custom tool modules | `POST /api/hooks`, `POST /api/sub-agents`, `GET /api/tools/`, `/api/tools/catalog` | llm, git, tmux, chromium | 32 | +| F27 | [Hooks, sub-agents and tool catalogs](F27-hooks-subagents-tools.md) | hooks and sub-agent catalogs, blocking hooks in a conversation, the tool registry and catalog, client tools, forwarded agent definitions, custom tool modules | `POST /api/hooks`, `POST /api/sub-agents`, `GET /api/tools/`, `/api/tools/catalog` | llm, git, tmux, chromium | 33 | | F28 | [Canvas apps: install, catalog and backend lifecycle](F28-canvas-extensions.md) | Canvas app install, catalog, enable, uninstall, bundle and icon; backend approval, prepare, start, stop, logs, data, failure states, restart | `/api/canvas-extensions/install`, `/api/canvas-extensions/installed[/{name}][/backend...]` | git, node; launch `--env TZ=UTC` | 30 | | F29 | [Canvas App backend bridge (session, HTTP and WebSocket proxy)](F29-app-backend-bridge.md) | the app-backend bridge: session cookies, origin and host checks, proxied HTTP methods and WebSockets, revocation | `/app-backends/{name}/session`, `/app-backends/{name}[/{path}]`, `WS /app-backends/{name}[/{path}]` | git; launch `--canvas-ingress` | 29 | | F30 | [OpenAI-compatible gateway (/v1)](F30-openai-gateway.md) | the OpenAI-compatible gateway: models, chat completions (plain and streaming), responses, conversation reuse, usage, the official OpenAI SDK | `GET /v1/models`, `POST /v1/chat/completions`, `/v1/responses` | llm, tmux | 28 | | F31 | [Persistence, restart recovery, leases and limits](F31-persistence-and-limits.md) | state that survives graceful and SIGKILL restarts, crash recovery, conversation ownership leases and takeover between servers, idle eviction, run capacity, config file vs environment precedence | behavior only: `restart`, `restart --hard`, a second server on shared storage | llm, tmux | 23 | | F32 | [Event webhooks and product telemetry](F32-webhooks-and-telemetry.md) | event webhooks (event batches, conversation updates, headers, buffering, flush, limits, retries) and product telemetry (off by default, exporter lifecycle, sanitized payloads, consent) | behavior only: `webhooks` / `OH_WEBHOOKS_*`, `OH_TELEMETRY_*` | llm; launch `--webhook-sink` | 30 | | F33 | [Python SDK remote client (RemoteConversation, RemoteWorkspace)](F33-sdk-remote-client.md) | the Python SDK in this checkout against the server: RemoteWorkspace commands, files, git and settings readers; RemoteConversation create, run, callbacks, streaming, reattach, fork, confirmation, pause, interrupt, stop hooks, client tools, completion logs | behavior only: `Workspace(host=...)`, `RemoteWorkspace`, `RemoteConversation` | llm, tmux, git | 43 | -| F34 | [TypeScript client (@openhands/typescript-client)](F34-typescript-client.md) | the TypeScript client against this checkout: server and version gate, workspace, bash, conversation, state, events and socket clients, hooks and MCP clients, endpoint audit and drift | behavior only: `@openhands/typescript-client` (`clients/typescript`) | llm, node, uvx, git, tmux, network | 32 | -| F37 | [Emoji reactions and feature flags](F37-reactions.md) | the `conversation_reactions` feature flag, the user's reactions over REST, validation, the agent's `react`/`unreact` tools, default targets | `GET /api/settings/experimental`, `PATCH /api/settings/experimental/{key}`, `POST /api/conversations/{id}/reactions` | llm | 7 | +| F34 | [TypeScript client (@openhands/typescript-client)](F34-typescript-client.md) | the TypeScript client against this checkout: server and version gate, workspace, bash, conversation, state, events and socket clients, hooks and MCP clients, endpoint audit and drift | behavior only: `@openhands/typescript-client` (`clients/typescript`) | llm, node, uvx, git, tmux, network | 33 | +| F37 | [Emoji reactions and feature flags](F37-reactions.md) | the `conversation_reactions` feature flag (stored and runtime env), the user's reactions over REST and the TypeScript client, validation, active-branch targets, the agent's `react`/`unreact` tools, what the model sees | `POST /api/conversations/{id}/reactions` (the flag's routes are F18's) | llm | 12 | ## Not mapped diff --git a/.agents/skills/verify-sdk/programs/S01/loop.py b/.agents/skills/verify-sdk/programs/S01/loop.py index c45332d8d9d..ea85267e10e 100644 --- a/.agents/skills/verify-sdk/programs/S01/loop.py +++ b/.agents/skills/verify-sdk/programs/S01/loop.py @@ -701,6 +701,7 @@ class Collides(BaseModel): rejected = {} for name, schema in (("reserved_name", Reserved), ("field_collision", Collides)): + probe = None try: bad = Agent( llm=make_llm([{"finish": "x"}]), @@ -712,9 +713,11 @@ class Collides(BaseModel): probe.send_message("x") probe.run() rejected[name] = False - probe.close() except Exception as exc: rejected[name] = type(exc).__name__ + finally: + if probe is not None: + probe.close() out["rejected"] = rejected elif mode == "context-overflow": diff --git a/.agents/skills/verify-sdk/programs/S02/fake_provider.py b/.agents/skills/verify-sdk/programs/S02/fake_provider.py index 30ad4a0201c..b27158db1ed 100644 --- a/.agents/skills/verify-sdk/programs/S02/fake_provider.py +++ b/.agents/skills/verify-sdk/programs/S02/fake_provider.py @@ -26,6 +26,13 @@ {"usage": {"prompt_tokens": 10, "completion_tokens": 5, "cached": 0}} {"encrypted": "..."} Responses: encrypted reasoning {"incomplete": true} Responses stream: no completed event + {"usage": {"cache_write": 600}} cache writes (chat: + prompt_tokens_details + .cache_creation_tokens; + Responses: input_tokens_details + .cache_write_tokens) + {"reasoning": "...", "stream_summary": true} Responses stream: summary + text deltas A request whose body has `"stream": true` is answered as server-sent events. """ @@ -208,7 +215,14 @@ def chat_reply(self, step: dict[str, Any]) -> dict[str, Any]: "prompt_tokens": usage["prompt_tokens"], "completion_tokens": usage["completion_tokens"], "total_tokens": usage["prompt_tokens"] + usage["completion_tokens"], - "prompt_tokens_details": {"cached_tokens": usage["cached"]}, + "prompt_tokens_details": { + "cached_tokens": usage["cached"], + **( + {"cache_creation_tokens": usage["cache_write"]} + if "cache_write" in usage + else {} + ), + }, }, } if step.get("empty"): @@ -415,7 +429,14 @@ def responses_reply(self, step: dict[str, Any]) -> dict[str, Any]: "input_tokens": usage["prompt_tokens"], "output_tokens": usage["completion_tokens"], "total_tokens": usage["prompt_tokens"] + usage["completion_tokens"], - "input_tokens_details": {"cached_tokens": usage["cached"]}, + "input_tokens_details": { + "cached_tokens": usage["cached"], + **( + {"cache_write_tokens": usage["cache_write"]} + if "cache_write" in usage + else {} + ), + }, "output_tokens_details": { "reasoning_tokens": 7 if step.get("reasoning") else 0 }, @@ -441,6 +462,21 @@ def responses_stream( }, ) ) + if item["type"] == "reasoning" and step.get("stream_summary"): + summary = item["summary"][0]["text"] + for i in range(0, len(summary), 4): + out.append( + ( + "response.reasoning_summary_text.delta", + { + "type": "response.reasoning_summary_text.delta", + "item_id": item["id"], + "output_index": index, + "summary_index": 0, + "delta": summary[i : i + 4], + }, + ) + ) if item["type"] == "message": text = item["content"][0]["text"] for i in range(0, len(text), 4): diff --git a/.agents/skills/verify-sdk/programs/S02/llm_calls.py b/.agents/skills/verify-sdk/programs/S02/llm_calls.py index 103adcad960..247f4b74af9 100644 --- a/.agents/skills/verify-sdk/programs/S02/llm_calls.py +++ b/.agents/skills/verify-sdk/programs/S02/llm_calls.py @@ -14,6 +14,9 @@ inline-image http image URLs inlined as data: URLs serialization dump, masking, round trip, model_copy, load_from_env features model feature table and the chat options sent per model + responses-stream a streamed Responses reply with a reasoning summary and + a message, through on_token; cache-write tokens on the + Responses and Chat Completions paths """ import asyncio @@ -438,6 +441,54 @@ async def collect(chunk) -> None: }, } +elif mode == "responses-stream": + step = { + "reasoning": "plan the reply", + "stream_summary": True, + "text": "the answer", + "usage": {"cached": 200, "cache_write": 600}, + } + deltas: list = [] + with FakeProvider([step], model="gpt-5-mini") as s: + llm = chat_llm( + s, + model="openai/gpt-5-mini", + api_mode="responses", + reasoning_summary="auto", + stream=True, + ) + r = llm.responses(MESSAGES, on_token=deltas.append) + usage = llm.metrics.accumulated_token_usage + summary_chunks = [] + for c in deltas: + for ch in getattr(c, "choices", None) or []: + d = ch.delta + summary_chunks.append( + { + "id": getattr(c, "id", None), + "content": getattr(d, "content", None), + "reasoning": getattr(d, "reasoning_content", None), + } + ) + out["deltas"] = summary_chunks + out["summary_as_reasoning"] = "".join(d["reasoning"] or "" for d in summary_chunks) + out["content_text"] = "".join(d["content"] or "" for d in summary_chunks) + out["chunk_ids"] = sorted({d["id"] for d in summary_chunks if d["id"]}) + out["final_text"] = text_of(r) + out["final_summary"] = list( + getattr(r.message, "responses_reasoning_item", None).summary + if getattr(r.message, "responses_reasoning_item", None) + else [] + ) + out["responses_cache_read"] = usage.cache_read_tokens if usage else None + out["responses_cache_write"] = usage.cache_write_tokens if usage else None + step = {"text": "hi", "usage": {"cached": 200, "cache_write": 600}} + with FakeProvider([step]) as s: + llm = chat_llm(s) + llm.completion(MESSAGES) + usage = llm.metrics.accumulated_token_usage + out["chat_cache_write"] = usage.cache_write_tokens if usage else None + elif mode == "features": from openhands.sdk.llm.utils.model_features import get_features @@ -448,6 +499,8 @@ async def collect(chunk) -> None: "anthropic/claude-opus-4-5-20251101", "litellm_proxy/anthropic/claude-sonnet-4-5-20250929", "deepseek/deepseek-chat", + "deepseek/deepseek-v4.1-flash", + "litellm_proxy/deepseek-v4.1-flash", "moonshot/kimi-k2-0711-preview", ] table = {} diff --git a/.agents/skills/verify-sdk/programs/S04/control.py b/.agents/skills/verify-sdk/programs/S04/control.py index 28ee26f2206..34ffb2eae42 100644 --- a/.agents/skills/verify-sdk/programs/S04/control.py +++ b/.agents/skills/verify-sdk/programs/S04/control.py @@ -70,11 +70,12 @@ def slow_llm(script, delay: float) -> SlowLLM: return llm -def conversation_for(llm, tools=("SleeperTool",), **kw): +def conversation_for(llm, tools=("SleeperTool",), callbacks=None, **kw): return Conversation( agent=Agent(llm=llm, tools=[Tool(name=t) for t in tools], **kw), workspace=WORKSPACE, visualizer=None, + callbacks=callbacks, ) @@ -270,30 +271,32 @@ async def drive() -> None: llm = TestLLM.from_messages( script_messages([batch, {"finish": "batch done"}]), usage_id="agent" ) - conversation = conversation_for(llm, tool_concurrency_limit=1) - conversation.send_message("Run the batch.") tokens: list = [] + holder: list = [] + + def record_token(_event) -> None: + # Events are emitted while a run is in progress, so the run's token is + # observable here however short the run is. + token = holder[0].cancel_token if holder else None + if token is not None and (not tokens or tokens[-1] is not token): + tokens.append(token) + + conversation = conversation_for( + llm, tool_concurrency_limit=1, callbacks=[record_token] + ) + holder.append(conversation) + conversation.send_message("Run the batch.") async def drive() -> None: await conversation.arun() - def watch_token() -> None: - while thread.is_alive(): - token = conversation.cancel_token - if token is not None and (not tokens or tokens[-1] is not token): - tokens.append(token) - time.sleep(0.02) - thread = threading.Thread(target=lambda: asyncio.run(drive())) thread.start() - watcher = threading.Thread(target=watch_token) - watcher.start() while count(conversation, ActionEvent) < 4 and thread.is_alive(): time.sleep(0.02) time.sleep(0.3) # call-0 is running conversation.interrupt() thread.join(timeout=60) - watcher.join() time.sleep(1.2) # let a worker that was already running finish first_token = tokens[0] if tokens else None out |= { @@ -305,11 +308,8 @@ def watch_token() -> None: } # The next run gets a fresh, uncancelled token and finishes the work. thread = threading.Thread(target=lambda: asyncio.run(drive())) - watcher = threading.Thread(target=watch_token) thread.start() - watcher.start() thread.join(timeout=60) - watcher.join() out |= { "tokens_seen": len(tokens), "second_token_is_new": len(tokens) >= 2 and tokens[-1] is not tokens[0], diff --git a/.agents/skills/verify-sdk/programs/S06/condense.py b/.agents/skills/verify-sdk/programs/S06/condense.py index 50c615a9ebf..4192fccb467 100644 --- a/.agents/skills/verify-sdk/programs/S06/condense.py +++ b/.agents/skills/verify-sdk/programs/S06/condense.py @@ -20,6 +20,9 @@ pipeline PipelineCondenser and NoOpCondenser inside conversations. resume a condensed conversation is reopened and keeps going from the summary. + dispositions get_observation_dispositions over a real conversation's + events plus a recovery AgentErrorEvent (recovery_reason= + "server_restart") ahead of a call's late result, against View. live the run's real model works through several commands with a real summarizing LLM (max_size=10), then is condensed on request. @@ -468,6 +471,65 @@ def summary_reached_agent(): out["resumed_status"] = reopened.state.execution_status.value reopened.close() +elif mode == "dispositions": + from openhands.sdk.context.view import get_observation_dispositions + from openhands.sdk.event import AgentErrorEvent, Event, ObservationBaseEvent + from openhands.sdk.event.error_classification import ( + ErrorClassification, + FailureKind, + ) + + script = [parallel_step(), *echo_steps(1), {"finish": "done"}] + conversation = conversation_for(script, None) + run(conversation) + events = list(conversation.state.events) + conversation.close() + actions = [e for e in events if isinstance(e, ActionEvent)] + first = actions[0] + real = next( + e + for e in events + if isinstance(e, ObservationBaseEvent) and e.tool_call_id == first.tool_call_id + ) + # What a restarted server persists for a call that was in flight: an + # outcome synthesized during recovery, ahead of the call's late result. + recovery = AgentErrorEvent( + error="A restart occurred while this tool was in progress.", + tool_name=first.tool_name, + tool_call_id=first.tool_call_id, + recovery_reason="server_restart", + classification=ErrorClassification(kind=FailureKind.INTERNAL, retryable=False), + ) + reloaded = Event.model_validate_json(recovery.model_dump_json()) + out["recovery_reason_roundtrip"] = getattr(reloaded, "recovery_reason", None) + out["recovery_reason_default"] = AgentErrorEvent( + error="x", tool_name="t", tool_call_id="c" + ).recovery_reason + at = events.index(real) + merged = [*events[:at], reloaded, *events[at:]] + before = [e.model_dump_json() for e in merged] + dispositions = get_observation_dispositions(iter(merged)) + out["tool_calls"] = len(actions) + out["dispositions"] = len(dispositions) + by_call = {d.tool_call_id: d for d in dispositions} + d0 = by_call[first.tool_call_id] + out["first_canonical_is_recovery"] = d0.canonical_event_id == reloaded.id + out["first_out_of_view_is_real"] = d0.out_of_view_event_ids == (real.id,) + out["others_single"] = all( + not d.out_of_view_event_ids + for k, d in by_call.items() + if k != first.tool_call_id + ) + out["order_follows_input"] = [d.tool_call_id for d in dispositions] == [ + a.tool_call_id for a in actions + ] + view = View.from_events(merged) + out["view_observations_match"] = [ + e.id for e in view.events if isinstance(e, ObservationBaseEvent) + ] == [d.canonical_event_id for d in dispositions] + out["events_unchanged"] = [e.model_dump_json() for e in merged] == before + out["actions_only"] = len(get_observation_dispositions(actions)) + elif mode == "live": condenser = LLMSummarizingCondenser( llm=make_llm(usage_id="condenser"), max_size=10, keep_first=2 diff --git a/.agents/skills/verify-sdk/programs/S08/store.py b/.agents/skills/verify-sdk/programs/S08/store.py index b061c564a4e..2b6884f91ca 100644 --- a/.agents/skills/verify-sdk/programs/S08/store.py +++ b/.agents/skills/verify-sdk/programs/S08/store.py @@ -11,6 +11,9 @@ traversal, a non-kebab name, requested/resolved refs, and an untracked directory picked up by list_installed. legacy a .installed.json with the old `plugins` and `skills` keys. + resume an installed plugin (skill and .mcp.json server) loads into a + persisted conversation; the plugin is disabled and the same + conversation is resumed with its persisted agent and sent a message. """ import json @@ -18,11 +21,11 @@ import sys from pathlib import Path -from build import commit, git_repo, head, skill_md, write +from build import claude_code_plugin, commit, git_repo, head, skill_md, write from recorder import recording_llm from sdk_verify import emit, run_paths -from openhands.sdk import Agent, Conversation +from openhands.sdk import Agent, Conversation, LocalConversation from openhands.sdk.extensions.installation import InstallationManager from openhands.sdk.plugin import ( disable_plugin, @@ -224,4 +227,48 @@ def rec(name, path, enabled=True): out["loaded"] = sorted(p.name for p in load_installed_plugins(installed_dir=store)) out["keys_after"] = sorted(metadata(store)) +elif mode == "resume": + plug = claude_code_plugin(run / "src" / "resplug", "resplug") + install_plugin(str(plug)) + base = run / "conversations" + + def open_conv(cid=None): + common = { + "workspace": str(workspace), + "persistence_dir": str(base), + "visualizer": None, + } + if cid is None: + agent = Agent(llm=recording_llm([{"finish": "done"}] * 2), tools=[]) + return Conversation(agent=agent, **common) + # Resume with the persisted agent: passing a new Agent would replace it + # and reset the ambient-plugin records, hiding whether the drop works. + return LocalConversation(agent=None, conversation_id=cid, **common) + + def snapshot(conv) -> dict: + ctx = conv.agent.agent_context + return { + "skills": sorted(s.name for s in ctx.skills) if ctx else [], + "mcp_servers": sorted(conv.agent.mcp_config or {}), + "recorded_skills": conv.state.ambient_plugin_skills, + "recorded_mcp": conv.state.ambient_plugin_mcp_servers, + } + + conv = open_conv() + cid = conv.id + conv.send_message("hi") + out["enabled"] = snapshot(conv) + conv.close() + out["disable"] = disable_plugin("resplug") + resumed = open_conv(cid) + out["resumed_same_id"] = resumed.id == cid + resumed.send_message("again") + out["after_disable"] = snapshot(resumed) + resumed.close() + enable_plugin("resplug") + again = open_conv(cid) + again.send_message("and again") + out["after_enable"] = snapshot(again) + again.close() + emit(out) diff --git a/.agents/skills/verify-sdk/programs/S11/task_secrets.py b/.agents/skills/verify-sdk/programs/S11/task_secrets.py new file mode 100644 index 00000000000..1128e2e2e97 --- /dev/null +++ b/.agents/skills/verify-sdk/programs/S11/task_secrets.py @@ -0,0 +1,134 @@ +"""A parent conversation with a secret delegates a command that needs it. + +The parent registers `PROBE_TOKEN` with `conversation.update_secrets(...)`, +runs `echo probe=${PROBE_TOKEN:+set} value=$PROBE_TOKEN` itself, then delegates the same +command to the programmatic sub-agent `prober` (create), and once more to the +same task id (resume). Each terminal observation says `probe=set` when the +secret reached that shell and `probe=` when it did not, and prints the value +so that masking is visible: the parent's observation shows ``, +and the secret's value appears in no observation or task result. +""" + +import json + +from s11_helpers import scripted, task_observations +from sdk_verify import emit, run_paths + +from openhands.sdk import Agent, Conversation, Tool +from openhands.sdk.subagent import register_agent +from openhands.tools.task import TaskToolSet +from openhands.tools.terminal import TerminalTool + + +SECRET = "s3cr3t-probe-value-6724" +PROBE = "echo probe=${PROBE_TOKEN:+set} value=$PROBE_TOKEN" + +paths = run_paths() +workspace = paths["workspace"] + +prober_scripts = [ + [ + {"tool": "terminal", "args": {"command": PROBE}}, + {"finish": "probed"}, + ], + [ + {"tool": "terminal", "args": {"command": PROBE}}, + {"finish": "probed again"}, + ], +] +sub_conversations = [] + + +def make_prober(_llm): + return Agent( + llm=scripted(prober_scripts.pop(0), usage_id="prober"), + tools=[Tool(name=TerminalTool.name)], + ) + + +register_agent("prober", make_prober, "Runs a probe command.") + +parent = Conversation( + agent=Agent( + llm=scripted( + [ + {"tool": "terminal", "args": {"command": PROBE}}, + { + "tool": "task", + "args": {"prompt": "Run the probe", "subagent_type": "prober"}, + }, + { + "tool": "task", + "args": { + "prompt": "Run the probe again", + "subagent_type": "prober", + "resume": "task_00000001", + }, + }, + {"finish": "done"}, + ], + usage_id="parent", + ), + tools=[Tool(name=TerminalTool.name), Tool(name=TaskToolSet.name)], + ), + workspace=str(workspace), + persistence_dir=str(paths["conversations"]), + max_iteration_per_run=20, + visualizer=None, +) +parent.update_secrets({"PROBE_TOKEN": SECRET}) +parent.send_message("Probe the token here and in a sub-agent.") +parent.run() + +parent_probe = None +for e in parent.state.events: + obs = getattr(e, "observation", None) + if obs is not None and type(obs).__name__ == "TerminalObservation": + parent_probe = obs.text + break +tasks = task_observations(parent.state.events) +status = parent.state.execution_status.value +parent.close() + + +def probe_value(text): + if text is None: + return None + if "probe=set" in text: + return "set" + if "probe=" in text: + return "unset" + return "missing" + + +# The sub-agents' own terminal output is in their persisted event logs. +sub_probes = [] +sub_texts = [] +sub_root = paths["conversations"] +for f in sorted(sub_root.rglob("subagents/*/events/*.json")): + event = json.loads(f.read_text()) + obs = event.get("observation") or {} + if event.get("kind") != "ObservationEvent" or obs.get("kind") not in ( + "TerminalObservation", + None, + ): + continue + text = "".join(c.get("text", "") for c in obs.get("content") or []) + if text.startswith("probe="): + sub_probes.append(probe_value(text)) + sub_texts.append(text) + +all_text = (parent_probe or "") + "".join(t["text"] for t in tasks) + "".join(sub_texts) +emit( + { + "ok": True, + "execution_status": status, + "parent_probe": probe_value(parent_probe), + "task_statuses": [t["status"] for t in tasks], + "sub_probes": sub_probes, + "create_probe": sub_probes[0] if sub_probes else None, + "resume_probe": sub_probes[1] if len(sub_probes) > 1 else None, + "parent_masked": "" in (parent_probe or ""), + "secret_leaked": SECRET in all_text, + } +) diff --git a/.agents/skills/verify-sdk/programs/S15/close_tab.py b/.agents/skills/verify-sdk/programs/S15/close_tab.py new file mode 100644 index 00000000000..bfd6d613b7e --- /dev/null +++ b/.agents/skills/verify-sdk/programs/S15/close_tab.py @@ -0,0 +1,62 @@ +"""Close a tab that is not focused, then list the tabs at once. + +A `BrowserToolExecutor` (the browser tool set's executor, called the way the +agent's tool calls reach it) opens the second page of the local site in a new +tab, switches back to the first tab, closes the new one and lists the tabs +immediately, five times. Reports how many of those lists still showed the +closed tab, and whether every close answered "Closed tab". +""" + +import json + +from S15.site import install_playwright_chromium, serve +from sdk_verify import emit + +from openhands.tools.browser_use import ( + BrowserCloseTabAction, + BrowserListTabsAction, + BrowserNavigateAction, + BrowserSwitchTabAction, +) +from openhands.tools.browser_use.impl import BrowserToolExecutor + + +ROUNDS = 5 + +install_playwright_chromium() +base = serve() +executor = BrowserToolExecutor(headless=True) + + +def tab_ids() -> list[str]: + return [t["tab_id"] for t in json.loads(executor(BrowserListTabsAction()).text)] + + +stale = 0 +closes: list[str] = [] +setup_ok = True +try: + executor(BrowserNavigateAction(url=f"{base}/index.html")) + for _ in range(ROUNDS): + executor(BrowserNavigateAction(url=f"{base}/page2.html", new_tab=True)) + before = tab_ids() + if len(before) != 2: + setup_ok = False + break + executor(BrowserSwitchTabAction(tab_id=before[0])) + closes.append(executor(BrowserCloseTabAction(tab_id=before[1])).text) + if tab_ids() != [before[0]]: + stale += 1 +finally: + executor.close() + +emit( + { + "ok": True, + "rounds": ROUNDS, + "setup_ok": setup_ok, + "all_closed": len(closes) == ROUNDS + and all(c.startswith("Closed tab") for c in closes), + "stale_lists": stale, + } +) diff --git a/.agents/skills/verify-sdk/programs/S16/reactions.py b/.agents/skills/verify-sdk/programs/S16/reactions.py new file mode 100644 index 00000000000..a42bbf5d0e3 --- /dev/null +++ b/.agents/skills/verify-sdk/programs/S16/reactions.py @@ -0,0 +1,197 @@ +"""Emoji reactions in an SDK program: the agent's `react`/`unreact` tools and +the user's `conversation.react(...)`. + +Turn 1 (user: "Please look into the build."): the scripted agent reacts +`eyes` (no message id, so the user's latest message), tries an emoji outside +the set, takes `eyes` back, reacts `white_check_mark` and finishes. The user +then reacts `heart` without a target (it goes on the agent's answer) and tries +an emoji outside the set. Turn 2 (user: "Thanks, that's all."): the agent +finishes. A recording LLM keeps the text each call was sent, so the program +reports whether the agent saw the user's reaction and its own. + +With the argument `live` the run's real model gets the two tools and one +message asking it to react first and then answer; the program reports the +reactions it left. +""" + +import sys + +from sdk_verify import emit, make_llm, run_paths, script_messages + +from openhands.sdk import Agent, Conversation, Tool +from openhands.sdk.event import ActionEvent, MessageEvent, ReactionEvent +from openhands.sdk.testing import TestLLM +from openhands.tools.reaction import ReactTool, UnreactTool + + +SEEN: list[str] = [] + + +class RecordingLLM(TestLLM): + def completion(self, messages, *args, **kwargs): # type: ignore[override] + SEEN.append( + "\n".join( + getattr(c, "text", "") for m in messages for c in (m.content or []) + ) + ) + return super().completion(messages, *args, **kwargs) + + +paths = run_paths() + + +def live() -> None: + conversation = Conversation( + agent=Agent( + llm=make_llm(), + tools=[Tool(name=ReactTool.name), Tool(name=UnreactTool.name)], + ), + workspace=str(paths["workspace"]), + persistence_dir=str(paths["conversations"]), + max_iteration_per_run=6, + visualizer=None, + ) + conversation.send_message( + "First react to this message with the eyes emoji using the react tool." + " Then answer: what is 6 times 7? Reply with just the number." + ) + message_id = next( + e.id + for e in conversation.state.events + if isinstance(e, MessageEvent) and e.source == "user" + ) + conversation.run() + events = list(conversation.state.events) + emit( + { + "ok": True, + "execution_status": conversation.state.execution_status.value, + "agent_reactions": [ + { + "emoji": e.emoji, + "removed": e.removed, + "on_message": e.target_event_id == message_id, + } + for e in events + if isinstance(e, ReactionEvent) and e.source == "agent" + ], + "react_calls": sum( + 1 + for e in events + if isinstance(e, ActionEvent) and e.tool_name == "react" + ), + } + ) + conversation.close() + + +if sys.argv[1:] == ["live"]: + live() + raise SystemExit(0) + +FIRST = "Please look into the build." +llm = RecordingLLM.from_messages( + script_messages( + [ + {"tool": "react", "args": {"emoji": "eyes"}}, + {"tool": "react", "args": {"emoji": "rocket"}}, + {"tool": "unreact", "args": {"emoji": "eyes"}}, + {"tool": "react", "args": {"emoji": "white_check_mark"}}, + {"finish": "The build is green."}, + {"finish": "You're welcome."}, + ] + ), + usage_id="agent", +) +conversation = Conversation( + agent=Agent( + llm=llm, + tools=[Tool(name=ReactTool.name), Tool(name=UnreactTool.name)], + ), + workspace=str(paths["workspace"]), + persistence_dir=str(paths["conversations"]), + visualizer=None, +) +conversation.send_message(FIRST) +first_msg_id = next( + e.id + for e in conversation.state.events + if isinstance(e, MessageEvent) and e.source == "user" +) +conversation.run() +status_1 = conversation.state.execution_status.value + +user_reaction = conversation.react("heart") +try: + conversation.react("rocket") + bad_user_emoji = "no error" +except ValueError as exc: + bad_user_emoji = str(exc)[:200] +status_after_react = conversation.state.execution_status.value + +conversation.send_message("Thanks, that's all.") +conversation.run() +status_2 = conversation.state.execution_status.value + +events = list(conversation.state.events) +finish_ids = [ + e.id for e in events if isinstance(e, ActionEvent) and e.tool_name == "finish" +] +reactions = [ + { + "source": e.source, + "emoji": e.emoji, + "removed": e.removed, + "on_first_message": e.target_event_id == first_msg_id, + "on_first_finish": bool(finish_ids) and e.target_event_id == finish_ids[0], + "text_ending": e.target_text_ending, + } + for e in events + if isinstance(e, ReactionEvent) +] +kinds = [type(e).__name__ for e in events] +agent_reaction_preceded_by = [ + kinds[i - 1] + for i, e in enumerate(events) + if isinstance(e, ReactionEvent) and e.source == "agent" and i > 0 +] +first_reaction_at = kinds.index("ReactionEvent") if "ReactionEvent" in kinds else -1 +observations = [ + { + "tool": getattr(e, "tool_name", None), + "is_error": getattr(e.observation, "is_error", None), + "text": getattr(e.observation, "text", "")[:200], + } + for e in events + if type(e).__name__ == "ObservationEvent" +] +errors = [ + getattr(e, "error", "")[:200] + for e in events + if type(e).__name__ == "AgentErrorEvent" +] +turn2_text = SEEN[-1] if SEEN else "" +conversation.close() +emit( + { + "ok": True, + "status_turn1": status_1, + "status_after_user_react": status_after_react, + "status_turn2": status_2, + "reactions": reactions, + "agent_reaction_preceded_by": agent_reaction_preceded_by, + "event_before_first_reaction": kinds[first_reaction_at - 1] + if first_reaction_at > 0 + else None, + "observations": observations, + "agent_errors": errors, + "bad_user_emoji": bad_user_emoji, + "user_reaction_target_is_finish": bool(finish_ids) + and user_reaction.target_event_id == finish_ids[0], + "turn2_sees_user_heart": "[The user reacted \u2764\ufe0f to this message:]\n" + "The build is green." in turn2_text, + "turn2_user_reaction_lines": turn2_text.count("[The user reacted"), + "turn2_sees_own_reaction_line": "[You " in turn2_text, + "llm_calls": len(SEEN), + } +) diff --git a/.agents/skills/verify-sdk/programs/S17/schema_constraints.py b/.agents/skills/verify-sdk/programs/S17/schema_constraints.py new file mode 100644 index 00000000000..80e390af6d6 --- /dev/null +++ b/.agents/skills/verify-sdk/programs/S17/schema_constraints.py @@ -0,0 +1,75 @@ +"""The JSON Schema keywords an `Action` declares, as the model is sent them. + +`IssueRefundAction` declares a `pattern` on `order_id`, a one-value +`Literal` (`const`) on `currency`, and `extra="forbid"` +(`additionalProperties: false`). The program renders the tool the way the +agent sends it (`to_openai_tool()`, `to_responses_tool()`) and reports which +of those keywords survive, next to what pydantic's own schema has. +""" + +from collections.abc import Sequence +from typing import Literal + +from pydantic import ConfigDict, Field +from sdk_verify import emit + +from openhands.sdk import Action, Observation, ToolDefinition +from openhands.sdk.tool import ToolExecutor + + +class IssueRefundAction(Action): + model_config = ConfigDict(extra="forbid") + order_id: str = Field(pattern=r"^ORD-[0-9]{6}$", description="Order id") + amount: int = Field(description="Amount in cents") + currency: Literal["USD"] = Field(description="Currency") + + +class RefundObservation(Observation): + pass + + +class RefundExecutor(ToolExecutor[IssueRefundAction, RefundObservation]): + def __call__(self, action, conversation=None): # noqa: ARG002 + return RefundObservation.from_text("ok") + + +class IssueRefundTool(ToolDefinition[IssueRefundAction, RefundObservation]): + @classmethod + def create(cls, conv_state=None, **params) -> Sequence["IssueRefundTool"]: # noqa: ARG003 + return [ + cls( + description="Issue a refund.", + action_type=IssueRefundAction, + observation_type=RefundObservation, + executor=RefundExecutor(), + ) + ] + + +tool = IssueRefundTool.create()[0] +pyd = IssueRefundAction.model_json_schema() +openai_params = tool.to_openai_tool()["function"]["parameters"] +responses_params = tool.to_responses_tool()["parameters"] + + +def keywords(params): + props = params.get("properties", {}) + currency = props.get("currency", {}) + return { + "order_id_pattern": props.get("order_id", {}).get("pattern"), + "currency_const": currency.get("const") + or (currency.get("enum") if len(currency.get("enum") or []) == 1 else None), + "additional_properties": params.get("additionalProperties"), + } + + +emit( + { + "ok": True, + "pydantic": keywords(pyd), + "openai": keywords(openai_params), + "responses": keywords(responses_params), + "openai_order_id": openai_params.get("properties", {}).get("order_id"), + "openai_currency": openai_params.get("properties", {}).get("currency"), + } +) diff --git a/.agents/skills/verify-sdk/programs/S21/acp_effort.py b/.agents/skills/verify-sdk/programs/S21/acp_effort.py new file mode 100644 index 00000000000..b83c90b0f8d --- /dev/null +++ b/.agents/skills/verify-sdk/programs/S21/acp_effort.py @@ -0,0 +1,234 @@ +"""Drive ACP reasoning effort the way a program does: `ACPAgent(acp_effort=...)`, +`Conversation.switch_acp_effort(...)` before and after the session starts, +across model switches, and through `discover_acp_models()`. The ACP server is +`fake_acp.py` with `FAKE_ACP_EFFORT=1`: model `m1` offers a `thought_level` +select (`effort`: low/medium/high, default medium) that a model switch resets, +and `m2` offers none. A server named `codex-acp` offers `reasoning_effort`. + +Usage: acp_effort.py +""" + +import json +import logging +import os +import sys +from pathlib import Path + +from sdk_verify import emit, run_paths + +from openhands.sdk import LLM, Agent, Conversation +from openhands.sdk.agent import ACPAgent + + +HERE = Path(__file__).resolve().parent +FAKE = [sys.executable, str(HERE / "fake_acp.py")] +paths = run_paths() +work = paths["run"] / "acp" / "effort" +work.mkdir(parents=True, exist_ok=True) +JOURNAL = work / "journal.jsonl" +os.environ["FAKE_ACP_JOURNAL"] = str(JOURNAL) +os.environ["FAKE_ACP_SESSIONS"] = str(work / "sessions.txt") +os.environ["FAKE_ACP_EFFORT"] = "1" +workspace = paths["workspace"] +workspace.mkdir(parents=True, exist_ok=True) +result: dict = {"ok": True} +warnings_seen: list[str] = [] + + +class _Capture(logging.Handler): + def emit(self, record: logging.LogRecord) -> None: + if record.levelno >= logging.WARNING: + warnings_seen.append(record.getMessage()[:300]) + + +logging.getLogger("openhands").addHandler(_Capture()) + + +def journal(kind: str) -> list[dict]: + if not JOURNAL.exists(): + return [] + rows = [json.loads(x) for x in JOURNAL.read_text().splitlines() if x.strip()] + return [r for r in rows if r["kind"] == kind] + + +def config_calls(since: int) -> list[str]: + return [ + f"{c['config_id']}={c['value']}" for c in journal("set_config_option")[since:] + ] + + +def run_turn(conversation, text: str) -> str | None: + conversation.send_message(text) + try: + conversation.run() + except Exception as exc: # recorded, the checks judge it + return f"{type(exc).__name__}: {str(exc)[:200]}" + return None + + +def error_of(fn, *args) -> str: + try: + fn(*args) + except Exception as exc: # recorded, the checks judge it + return f"{type(exc).__name__}: {exc}"[:300] + return "no error" + + +def view(conversation) -> dict: + agent = conversation.agent + return { + "acp_effort": agent.acp_effort, + "current_model_id": agent.current_model_id, + "current_effort": agent.current_effort, + "available_efforts": [o.value for o in agent.available_efforts], + "hint_effort": conversation.state.agent_state.get( + "acp_current_effort", "unset" + ), + "session_id": conversation.state.agent_state.get("acp_session_id"), + } + + +def persisted_agent(conversation) -> dict: + base = Path(conversation.state.persistence_dir or "") / "base_state.json" + data = json.loads(base.read_text()) + return { + "acp_effort": data["agent"].get("acp_effort"), + "hint_effort": data.get("agent_state", {}).get("acp_current_effort", "unset"), + } + + +# 1. Pinned at construction, then switched live. +calls = len(journal("set_config_option")) +live = Conversation( + agent=ACPAgent(acp_command=FAKE, acp_model="m1", acp_effort="high"), + workspace=str(workspace), + visualizer=None, + persistence_dir=str(work / "store-live"), +) +result["start_error"] = run_turn(live, "hello") +result["start"] = {**view(live), "config_calls": config_calls(calls)} +session = live.state.agent_state.get("acp_session_id") + +calls = len(journal("set_config_option")) +live.switch_acp_effort("low") +result["switched"] = { + **view(live), + "config_calls": config_calls(calls), + "same_session": live.state.agent_state.get("acp_session_id") == session, + "persisted": persisted_agent(live), +} +calls = len(journal("set_config_option")) +result["unoffered_error"] = error_of(live.switch_acp_effort, "ultra") +result["blank_error"] = error_of(live.switch_acp_effort, " ") +result["after_refused"] = {**view(live), "config_calls": config_calls(calls)} + +live.switch_acp_model("m2") +result["on_m2"] = view(live) +result["on_m2_error"] = error_of(live.switch_acp_effort, "low") +calls = len(journal("set_config_option")) +live.switch_acp_model("m1") +result["back_on_m1"] = {**view(live), "config_calls": config_calls(calls)} +prompts = len(journal("prompt")) +result["next_turn_error"] = run_turn(live, "again") +result["next_turn"] = { + "status": live.state.execution_status.value, + "prompts": len(journal("prompt")) - prompts, + "same_session": live.state.agent_state.get("acp_session_id") == session, +} +live.agent.close() +live.close() + +# 2. A level the model does not offer at session start is skipped, not fatal. +calls = len(journal("set_config_option")) +warn = len(warnings_seen) +skipped = Conversation( + agent=ACPAgent(acp_command=FAKE, acp_model="m1", acp_effort="ultra"), + workspace=str(workspace), + visualizer=None, +) +result["skipped_error"] = run_turn(skipped, "hello") +result["skipped"] = { + **view(skipped), + "status": skipped.state.execution_status.value, + "config_calls": config_calls(calls), + "warned": any("ultra" in w for w in warnings_seen[warn:]), +} +skipped.agent.close() +skipped.close() + +# 3. Before the first run the effort is only persisted; the session applies it. +starts = len(journal("start")) +calls = len(journal("set_config_option")) +early = Conversation( + agent=ACPAgent(acp_command=FAKE), + workspace=str(workspace), + visualizer=None, + persistence_dir=str(work / "store-early"), +) +early.switch_acp_effort("high") +result["deferred"] = { + **view(early), + "processes_started": len(journal("start")) - starts, + "persisted": persisted_agent(early), + "blank_error": error_of(early.switch_acp_effort, ""), +} +result["deferred_error"] = run_turn(early, "hello") +result["deferred_started"] = {**view(early), "config_calls": config_calls(calls)} +early.agent.close() +early.close() + +# 4. A legacy / id on a server the SDK knows (codex-acp): +# the model id is bare, and acp_effort wins over the suffix. +os.environ["FAKE_ACP_NAME"] = "codex-acp" +legacy = {} +for name, effort in (("suffix", None), ("explicit", "low")): + calls = len(journal("set_config_option")) + conversation = Conversation( + agent=ACPAgent(acp_command=FAKE, acp_model="gpt-5.5/high", acp_effort=effort), + workspace=str(workspace), + visualizer=None, + ) + error = run_turn(conversation, "hello codex") + legacy[name] = { + **view(conversation), + "error": error, + "config_calls": config_calls(calls), + } + conversation.agent.close() + conversation.close() +os.environ.pop("FAKE_ACP_NAME") +result["legacy"] = legacy + +# 5. Not an ACP conversation. +native = Conversation( + agent=Agent(llm=LLM(model="openai/qa-native", api_key="sk-qa"), tools=[]), + workspace=str(workspace), + visualizer=None, +) +result["native_error"] = error_of(native.switch_acp_effort, "high") +native.close() + +# 6. Discovery reports the effort per model; settings carry acp_effort. +from openhands.sdk.agent.acp_model_discovery import discover_acp_models # noqa: E402 +from openhands.sdk.settings import ACPAgentSettings # noqa: E402 + + +d = discover_acp_models( + ACPAgentSettings(acp_server="custom", acp_command=FAKE), work_root=str(work) +).model_dump(mode="json") +result["discovery"] = { + "error": d.get("error"), + "current_model_id": d.get("current_model_id"), + "current_effort": d.get("current_effort"), + "available_efforts": [o["value"] for o in d.get("available_efforts") or []], + "efforts_by_model": { + k: [o["value"] for o in v] for k, v in (d.get("efforts_by_model") or {}).items() + }, +} +settings = ACPAgentSettings(acp_server="custom", acp_command=FAKE, acp_effort="low") +result["settings_agent_effort"] = settings.create_agent().acp_effort +result["settings_roundtrip_effort"] = ACPAgentSettings.model_validate( + settings.model_dump(mode="json") +).acp_effort + +emit(result) diff --git a/.agents/skills/verify-sdk/programs/S21/fake_acp.py b/.agents/skills/verify-sdk/programs/S21/fake_acp.py index d4f46d5ab52..73126a9ae67 100644 --- a/.agents/skills/verify-sdk/programs/S21/fake_acp.py +++ b/.agents/skills/verify-sdk/programs/S21/fake_acp.py @@ -17,6 +17,11 @@ FAKE_ACP_KEY is set FAKE_ACP_INIT_SLEEP seconds to stall in initialize FAKE_ACP_LEGACY_MODELS report models in the UNSTABLE `models` block + FAKE_ACP_EFFORT m1 also offers a reasoning-effort select (id + `effort`, category `thought_level`, levels + low/medium/high, default medium) that a model + switch resets; m2 offers none. A server named + codex-* always offers `reasoning_effort` this way. What a prompt does depends on a tag in its first text block: (none) thought, two tool calls (the first asks permission and streams @@ -55,6 +60,20 @@ ids = itertools.count(1000) cancelled: set[str] = set() models: dict[str, str] = {} +efforts: dict[str, str] = {} +EFFORT_LEVELS = ("low", "medium", "high") +DEFAULT_EFFORT = "medium" + + +def effort_config_id(session_id: str) -> str | None: + """The reasoning-effort option the session's current model offers.""" + if NAME.startswith("codex"): + return "reasoning_effort" + if os.environ.get("FAKE_ACP_EFFORT") and models.get(session_id, "m1") == "m1": + return "effort" + return None + + costs: dict[str, float] = {} attempts: dict[str, int] = {} forks: set[str] = set() @@ -131,14 +150,16 @@ def model_options(session_id: str) -> list[dict]: "options": [{"value": m, "name": m.upper()} for m in MODELS], } ] - if NAME.startswith("codex"): + config_id = effort_config_id(session_id) + if config_id is not None: options.append( { - "id": "reasoning_effort", + "id": config_id, "name": "Effort", "type": "select", - "currentValue": "medium", - "options": [{"value": v, "name": v} for v in ("low", "medium", "high")], + "category": "thought_level", + "currentValue": efforts.get(session_id, DEFAULT_EFFORT), + "options": [{"value": v, "name": v.title()} for v in EFFORT_LEVELS], } ) return options @@ -434,6 +455,15 @@ def handle(message: dict) -> None: return if config_id == "model": models[session_id] = value + efforts.pop(session_id, None) # a model switch resets the effort + elif config_id == effort_config_id(session_id) and value in EFFORT_LEVELS: + efforts[session_id] = value + else: + respond( + request_id, + error={"code": -32602, "message": f"Unknown {config_id} {value}"}, + ) + return respond(request_id, {"configOptions": model_options(session_id)}) elif method == "session/set_model": session_id, value = params.get("sessionId"), params.get("modelId") diff --git a/.agents/skills/verify-sdk/references/feature-map/README.md b/.agents/skills/verify-sdk/references/feature-map/README.md index 3f2f315b2b9..fb982fec1e9 100644 --- a/.agents/skills/verify-sdk/references/feature-map/README.md +++ b/.agents/skills/verify-sdk/references/feature-map/README.md @@ -10,9 +10,9 @@ agent-server map and follows its layout. Family IDs here are `Snn`, so they never collide with that map's `Fnn`. -Maintenance baseline: main@4db1ad742ad143ddc3411631f04efb70323f40f9 (2026-11-09). The first mapping; the next maintenance pass starts from this commit, a pass proposes the next baseline in its PR, and merging that PR accepts it. +Maintenance baseline: main@fb75ae907585c1634a85fd5168a795dd48eb4d65 (2026-11-12). The next maintenance pass starts from this commit, a pass proposes the next baseline in its PR, and merging that PR accepts it. -The map has **458 sub-features**, **29 of them untested** (no unit or cross +The map has **464 sub-features**, **31 of them untested** (no unit or cross test exercises the behavior; most of those have an example script). ## What each sub-feature links @@ -76,22 +76,22 @@ an interactive login are labeled **blocked**. | S03 | [LLM registry, routing, fallback, profiles and model switching](S03-llm-routing-profiles.md) | `LLMRegistry`, multimodal and random routers, `FallbackStrategy`, `LLMProfileStore`, `switch_profile`/`switch_llm`, the switch-LLM tool, `route_task_to_model`, subscription login | `LLMRegistry`, `MultimodalRouter`, `FallbackStrategy`, `LLMProfileStore`, `LLM.subscription_login()` | llm, network | 24 | 3 | | S04 | [Local conversation lifecycle](S04-local-conversation.md) | factory and ids, persistence and resume, callbacks, close, pause, `arun` and interrupt, secrets, visualizers, fork and branches, goal loop, final response, stats, title and tags | `Conversation(persistence_dir=, callbacks=, visualizer=, secrets=)`, `run`/`pause`/`arun`/`interrupt`/`fork`/`run_goal` | llm, tmux, network, uvx | 25 | 0 | | S05 | [Events and the event log](S05-events.md) | event model, JSON round trip, conversion to LLM messages and OpenAI formats, unmatched actions, the `EventLog`, error classification, completion and token events | `Event.model_validate_json`, `events_to_messages`, `conversation.state.events` | llm, tmux | 18 | 1 | -| S06 | [Agent context, system prompt, condensers and memory](S06-context-condensers.md) | `AgentContext`, system prompt sections and vendor files, persona, persistent memory, the condenser `View`, `LLMSummarizingCondenser`, `condense()`, overflow condensation, no-op and pipeline condensers | `Agent(agent_context=, system_prompt=, condenser=)`, `AgentContext(load_memory=)`, `conversation.condense()` | llm, tmux | 23 | 1 | +| S06 | [Agent context, system prompt, condensers and memory](S06-context-condensers.md) | `AgentContext`, system prompt sections and vendor files, persona, persistent memory, the condenser `View`, `LLMSummarizingCondenser`, `condense()`, overflow condensation, no-op and pipeline condensers | `Agent(agent_context=, system_prompt=, condenser=)`, `AgentContext(load_memory=)`, `conversation.condense()` | llm, tmux | 24 | 1 | | S07 | [Skills](S07-skills.md) | skill sources (programmatic, Markdown, AGENTS.md, SKILL.md, user, project, public, installed), keyword, task and path triggers, ``, `invoke_skill`, inline commands, disabling, install and marketplace | `AgentContext(skills=, load_*_skills=, disabled_skills=)`, `.agents/skills/`, `install_skill` | llm, git, network | 24 | 0 | | S08 | [Plugins, marketplaces and the extension store](S08-plugins-marketplace.md) | both plugin layouts, path containment, commands as skills, merging into the agent, `resolved_plugins`, fetch cache, installed and ambient plugins, marketplace registry, `load_plugin()` | `Conversation(plugins=[PluginSource(...)])`, `Plugin.load`, `conversation.load_plugin()` | llm, git, network, uvx | 20 | 0 | | S09 | [MCP client and tools](S09-mcp.md) | stdio, HTTP and SSE servers, auth and OAuth, enabled flag, timeouts, failing servers, schema conversion, observations, secrets in calls, reconnection, `list_changed`, tool filtering | `Agent(mcp_config=, filter_tools_regex=)`, `create_mcp_tools`, `MCPOAuthAuthentication` | llm, uvx, node, network | 19 | 0 | | S10 | [Hooks](S10-hooks.md) | config format and discovery, matchers, command hooks, blocking tool calls and prompts, context injection, Stop, SessionStart and SessionEnd, `HookExecutionEvent`, prompt and agent hooks, plugin hooks | `Conversation(hook_config=HookConfig(...))`, `HookDefinition(type=)`, `HookManager` | llm, git | 20 | 1 | -| S11 | [Sub-agents and delegation](S11-subagents-delegation.md) | the `task` tool lifecycle, agent registry, Markdown agent files and `AgentDefinition`, built-in agents, `SubAgentScope`, the legacy delegate executor | `Tool(name=TaskToolSet.name)`, `register_agent`, `.agents/agents/*.md` | llm | 23 | 2 | +| S11 | [Sub-agents and delegation](S11-subagents-delegation.md) | the `task` tool lifecycle, agent registry, Markdown agent files and `AgentDefinition`, built-in agents, `SubAgentScope`, the legacy delegate executor | `Tool(name=TaskToolSet.name)`, `register_agent`, `.agents/agents/*.md` | llm | 24 | 3 | | S12 | [Critic and critic-driven refinement](S12-critic.md) | `CriticResult`, rule critics, evaluation modes, iterative refinement, `APIBasedCritic`, critic from settings | `Agent(critic=APIBasedCritic(...))`, `event.critic_result` | llm, network | 15 | 4 | | S13 | [Security analyzers, confirmation mode and secrets](S13-security-confirmation.md) | risk levels, confirmation policies, approve and reject, LLM, pattern, policy-rail, ensemble, ToolShield and GraySwan analyzers, secret sources, env injection, masking | `set_confirmation_policy`, `set_security_analyzer`, `Conversation(secrets=)` | llm, network | 21 | 0 | | S14 | [Core tools and tool presets](S14-core-tools.md) | terminal (tmux and subprocess), file editor, apply_patch, Gemini file tools, glob, grep, output masking, default, GPT-5 and Gemini presets | `Tool(name="terminal" \| "file_editor" \| ...)`, `get_default_agent`, `get_gpt5_*`, `get_gemini_*` | llm, tmux, git, network | 24 | 0 | | S15 | [Browser tools and session recording](S15-browser.md) | `BrowserToolSet`, Chromium detection and fallback, navigation, interaction, tabs, storage, screenshots, rrweb recording | `Tool(name="browser_tool_set")`, `BrowserToolExecutor` | chromium, llm, network | 17 | 0 | -| S16 | [Planning, task tracking, oracle, ToM and workflow tools](S16-planning-workflow-tools.md) | task tracker, planning preset and PLAN.md editor, plan-then-execute, `ask_oracle`, `tom_consult`, sleeptime compute, the workflow tool | `Tool(name="task_tracker" \| "ask_oracle" \| "tom_consult" \| "workflow_tool_set")`, `get_planning_agent` | llm | 20 | 6 | -| S17 | [Custom tools, client-defined tools and the tool registry](S17-custom-and-client-tools.md) | custom tool classes and tool sets, registration and naming, tool schemas, injected fields, catalog seal, module qualnames, client tools, `execute_tool` | `register_tool`, `Tool(name, params)`, `Conversation(client_tools=)`, `execute_tool` | llm | 23 | 0 | +| S16 | [Planning, task tracking, oracle, ToM, workflow and reaction tools](S16-planning-workflow-tools.md) | task tracker, planning preset and PLAN.md editor, plan-then-execute, `ask_oracle`, `tom_consult`, sleeptime compute, the workflow tool, emoji reactions (`react`/`unreact`, `conversation.react`) | `Tool(name="task_tracker" \| "ask_oracle" \| "tom_consult" \| "workflow_tool_set" \| "react" \| "unreact")`, `get_planning_agent`, `LocalConversation.react` | llm | 21 | 6 | +| S17 | [Custom tools, client-defined tools and the tool registry](S17-custom-and-client-tools.md) | custom tool classes and tool sets, registration and naming, tool schemas, injected fields, catalog seal, module qualnames, client tools, `execute_tool` | `register_tool`, `Tool(name, params)`, `Conversation(client_tools=)`, `execute_tool` | llm | 24 | 1 | | S18 | [Workspaces and sandbox backends](S18-workspace-backends.md) | the `Workspace` factory, `LocalWorkspace`, `RemoteWorkspace`, and the Docker, API-remote, Cloud, Apptainer and agent-sandbox backends (start, health, commands, files, pause, cleanup) | `Workspace(...)`, `LocalWorkspace`, `openhands.workspace.*Workspace` | docker, apptainer, cloud, k8s, network, llm, git | 24 | 3 | | S19 | [Agent settings, persisted schema migrations and agent profiles](S19-settings.md) | `OpenHandsAgentSettings`, ACP settings, schema migrations v0 to v8, settings diffs and export, the persisted-settings compatibility gate, agent profiles and their store | `openhands.sdk.settings`, `openhands.sdk.profiles`, `~/.openhands/agent-profiles/` | llm, tmux, network | 25 | 0 | | S20 | [Observability, tracing and logging](S20-observability.md) | enabling tracing, Laminar and OTLP backends, conversation, step, tool and LLM spans, sub-agent trace links, automation helpers, the logger | `LMNR_*`/`OTEL_*` env vars, `Conversation(observability_*=)`, `get_logger` | llm, network | 20 | 5 | -| S21 | [ACP agent (in-process)](S21-acp-agent.md) | construction, provider registry, handshake, turns and tool-call events, permission auto-approval, timeouts and cancel, session resume, model switching, usage, MCP forwarding, secrets, cleanup | `ACPAgent(acp_command=...)`, `switch_acp_model`, `ACPAgentSettings` | node, network, llm | 24 | 2 | +| S21 | [ACP agent (in-process)](S21-acp-agent.md) | construction, provider registry, handshake, turns and tool-call events, permission auto-approval, timeouts and cancel, session resume, model switching, usage, MCP forwarding, secrets, cleanup | `ACPAgent(acp_command=...)`, `switch_acp_model`, `ACPAgentSettings` | node, network, llm | 26 | 2 | ## Not mapped diff --git a/.agents/skills/verify-sdk/references/feature-map/S01-agent-loop.md b/.agents/skills/verify-sdk/references/feature-map/S01-agent-loop.md index 59b7eea7ff7..0e735368a07 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S01-agent-loop.md +++ b/.agents/skills/verify-sdk/references/feature-map/S01-agent-loop.md @@ -119,7 +119,7 @@ Scripted bullets use the SDK's `TestLLM` (`--script`, or `make_llm(script=...)` --check finish_over_budget.status eq finished --check finish_over_budget.codes len-eq 0 \ --check factory_accepts_budget eq true ``` - Fixed: a known failure on 2026-10-13 (the public `Conversation(...)` factory refused `max_budget_per_run` with a `TypeError`); the factory takes it since #6257, and the bullet passes again on 2026-11-03. + Expected: the public `Conversation(...)` factory accepts `max_budget_per_run` (`factory_accepts_budget` is true). - **Stuck on a repeating action (`S01.stuck-detect`, `S01.stuck-recover`).** Five identical `ls` calls: the fourth pair sets STUCK; a new message moves it to IDLE and the run then reaches the script's `finish`. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -135,7 +135,7 @@ Scripted bullets use the SDK's `TestLLM` (`--script`, or `make_llm(script=...)` control-sdk conversation send "$CID" --text "List files" --run --check execution_status eq stuck control-sdk conversation run "$CID" --check script_left lt 2 ``` -Expected: `run()` on a detector-STUCK conversation makes at least one more LLM call. Before #6648, the detector still saw the repeated pairs since the last user message and set STUCK again before any LLM call (`script_left` stayed 2); resuming now starts a fresh detection window, as a new user message does. + Expected: `run()` on a detector-STUCK conversation makes at least one more LLM call: resuming starts a fresh detection window, as a new user message does. - **Custom thresholds (`S01.stuck-thresholds`).** Three identical `ls` calls with the defaults (no stuck) and with `stuck_detection_thresholds={"action_observation": 2}`. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -265,6 +265,7 @@ Expected: `run()` on a detector-STUCK conversation makes at least one more LLM c ## Gotchas +- A rejected `response_schema` on a terminal tool raises `ValueError` after the terminal has already started its tmux server, and nothing closes it: `close()` leaves the server running after the program exits ([#6748](https://github.com/OpenHands/software-agent-sdk/issues/6748)). `S01.structured-output` passes but leaves two such servers per run; end them with `TMUX_TMPDIR= tmux -L kill-server`. - `01_hello_world.py` is in `_EXCLUDED_EXAMPLES` in `tests/examples/test_examples.py`, so the `test-examples` label never runs it; the other seven examples here do run there (`pytest tests/examples/test_examples.py --run-examples`). - All examples read `LLM_API_KEY`, `LLM_MODEL`, `LLM_BASE_URL`; `56_structured_output.py` defaults `LLM_MODEL` to `anthropic/claude-sonnet-4-5-20250929`, the rest to `gpt-5.5`. `17_image_input.py` asserts `llm.vision_is_active()` and fails fast on a text-only model; it also fetches an image from python.org. - Examples run with the repo root as workspace and write files there (`FACTS.txt`, `image_report.md`, the document from example 18). diff --git a/.agents/skills/verify-sdk/references/feature-map/S02-llm-core.md b/.agents/skills/verify-sdk/references/feature-map/S02-llm-core.md index 3d9a351933f..43f339aebb6 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S02-llm-core.md +++ b/.agents/skills/verify-sdk/references/feature-map/S02-llm-core.md @@ -96,6 +96,21 @@ The scripted bullets point a real `LLM` (`openai/...`, `anthropic/...`) at `prog --check sync_callback.tool_arguments.0 eq '{"q": 1}' ``` Content, reasoning and tool-call deltas reach the callback and the final message is reassembled (tool arguments and cache-read tokens included); `stream=True` without a callback sends a plain request and returns the full reply. +- **A streamed Responses reasoning summary (`S02.streaming`, `S02.responses-reasoning`).** `openai/gpt-5-mini` with `api_mode="responses"`, `reasoning_summary="auto"` and `stream=True`; the reply streams a reasoning item (`response.reasoning_summary_text.delta` events) and then a message, and `llm.responses(..., on_token=...)` collects the deltas. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --field run) + control-sdk program S02/llm_calls --arg responses-stream \ + --check final_text eq "the answer" --check final_summary.0 eq "plan the reply" --check content_text eq "the answer" \ + --check summary_as_reasoning eq "plan the reply" + ``` + Expected: the summary reaches the callback as `delta.reasoning_content`, as chat-path reasoning does, and the answer as `delta.content`; the final message carries both. Known failure (reproduced 2026-11-12, [#6695](https://github.com/OpenHands/software-agent-sdk/issues/6695)): no summary delta reaches the callback at all (`summary_as_reasoning` is empty; every delta is the message's `content`, chunk id `msg_1`). With the installed LiteLLM (1.105.1) a `response.reasoning_summary_text.delta` event is parsed as a `GenericEvent`, never the `ReasoningSummaryTextDeltaEvent` that `LLM._process_stream_event` looks for, and that branch would emit it as `delta.content` rather than `delta.reasoning_content`. The final message still has the summary. +- **Cache-write tokens on the Responses path (`S02.metrics`).** The same program: the Responses reply reports `input_tokens_details.cache_write_tokens: 600`, and a Chat Completions reply reports `prompt_tokens_details.cache_creation_tokens: 600`. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --field run) + control-sdk program S02/llm_calls --arg responses-stream \ + --check chat_cache_write eq 600 --check responses_cache_read eq 200 --check responses_cache_write eq 600 + ``` + Expected: `TokenUsage.cache_write_tokens` is 600 on both paths. Known failure (reproduced 2026-11-12, [#6691](https://github.com/OpenHands/software-agent-sdk/issues/6691)): the Responses path records 0 (`normalize_usage` reads only `cached_tokens` from `input_tokens_details`); the chat path records 600. - **Timeouts (`S02.timeouts`).** A reply slower than `timeout=1`, a stream that stalls longer than `stream_idle_timeout=1` on `acompletion`, and the same stall inside `await conversation.arun()`. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -123,12 +138,14 @@ The scripted bullets point a real `LLM` (`openai/...`, `anthropic/...`) at `prog --check features.anthropic/claude-sonnet-4-5-20250929.thinking_mode eq manual \ --check features.anthropic/claude-sonnet-4-5-20250929.prompt_cache eq true \ --check features.deepseek/deepseek-chat.vision eq false --check proxy_prefix_stripped eq true \ + --check 'features.deepseek/deepseek-v4\.1-flash.vision' eq true --check 'features.deepseek/deepseek-v4\.1-flash.force_string' eq false \ + --check 'features.litellm_proxy/deepseek-v4\.1-flash.vision' eq true \ --check sent.gpt5.reasoning_effort eq high --check sent.gpt5.temperature missing --check sent.gpt5.top_p missing \ --check sent.gpt4o.reasoning_effort missing --check sent.gpt4o.temperature eq 0.5 --check sent.gpt4o.top_p eq 0.9 \ --check sent.azure.max_tokens eq true --check sent.azure.max_completion_tokens eq false \ --check sent.openrouter.openrouter_headers eq true ``` - A reasoning model gets `reasoning_effort` (default `high`) and loses `temperature`/`top_p`; a non-reasoning model keeps them; Azure gets `max_tokens`. (Prompt-cache keys show in the Responses bullet above.) + A reasoning model gets `reasoning_effort` (default `high`) and loses `temperature`/`top_p`; a non-reasoning model keeps them; Azure gets `max_tokens`. `deepseek-v4.1-flash`, which LiteLLM's registry does not mark as visual, is vision-capable from the SDK's own list, under a proxy prefix too, and keeps list content. (Prompt-cache keys show in the Responses bullet above.) - **Prompted tool calling (`S02.prompted-tool-calling`, `S02.native-tool-calling`).** `native_tool_calling=False`: the agent's tools are described in the prompt and the model's `` text becomes real tool calls. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -161,7 +178,7 @@ The scripted bullets point a real `LLM` (`openai/...`, `anthropic/...`) at `prog --check chat.text eq hi --check chat.temperatures.1 eq 1.0 \ --check chat_exhausted.error eq LLMNoResponseError ``` - Known failure (reproduced 2026-11-03, not yet filed; it moved with the LiteLLM 1.93 to 1.105 upgrade): the chat path now behaves as documented, but on the Responses path a stream that ends without its completed event no longer reaches the SDK's `LLMNoResponseError` check. LiteLLM raises `APIConnectionError` first, the SDK retries it as a transient error without the temperature bump (`responses.temperatures` is `[0.0, 0.0]`), and an exhausted async call ends in `LLMServiceUnavailableError`. On 2026-10-13, with LiteLLM 1.93, it was the other way round: the Responses path worked and a chat reply with `"choices": []` became `LLMServiceUnavailableError`. The unit tests mock the LiteLLM call, so they meet neither. + Known failure (reproduced 2026-11-12, [#6657](https://github.com/OpenHands/software-agent-sdk/issues/6657)): the chat path now behaves as documented, but on the Responses path a stream that ends without its completed event no longer reaches the SDK's `LLMNoResponseError` check. LiteLLM raises `APIConnectionError` first, the SDK retries it as a transient error without the temperature bump (`responses.temperatures` is `[0.0, 0.0]`), and an exhausted async call ends in `LLMServiceUnavailableError`. The unit tests mock the LiteLLM call, so they do not meet it. - **Error mapping (`S02.error-mapping`).** Each provider error body, as an OpenAI-compatible endpoint sends it, and a dead endpoint. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -259,9 +276,12 @@ The scripted bullets point a real `LLM` (`openai/...`, `anthropic/...`) at `prog export SDK_VERIFY_RUN=$(control-sdk new --llm deepseek-flash --field run) control-sdk program S02/api_compliance \ --check patterns len-eq 8 --check rejections_typed eq true \ - --check results.unmatched_tool_use.error_type eq LLMMalformedConversationHistoryError + --check results.unmatched_tool_use.error_type eq LLMMalformedConversationHistoryError \ + --check results.unmatched_tool_result.error_type eq LLMMalformedConversationHistoryError \ + --check results.duplicate_tool_call_id.error_type eq LLMMalformedConversationHistoryError \ + --check results.parallel_wrong_order.error_type eq LLMMalformedConversationHistoryError ``` - On 2026-11-03 DeepSeek rejected all eight. Five came back as `LLMMalformedConversationHistoryError`; the three whose provider message is "Messages with role 'tool' must be a response to a preceding message with 'tool_calls'" (unmatched tool result, duplicate id, parallel results in the wrong order) came back as the generic `LLMBadRequestError`, so a caller that recovers from malformed histories by catching the specific error misses them. + Known failure (reproduced 2026-11-12, [#6658](https://github.com/OpenHands/software-agent-sdk/issues/6658)): DeepSeek rejects all eight. Five come back as `LLMMalformedConversationHistoryError`; the three whose provider message is "Messages with role 'tool' must be a response to a preceding message with 'tool_calls'" (unmatched tool result, duplicate id, parallel results in the wrong order) come back as the generic `LLMBadRequestError`, so a caller that recovers from malformed histories by catching the specific error misses them. - **Examples 22 and 23 and the thinking-block condenser test (`S02.anthropic-thinking`, `S02.responses-reasoning`), blocked.** Example 22 and `c01_thinking_block_condenser` need an Anthropic extended-thinking model and key; example 23 hardcodes `openhands/gpt-5-mini-2025-08-07` and needs an OpenHands provider key. The run's DeepSeek key reaches neither. ## Gotchas diff --git a/.agents/skills/verify-sdk/references/feature-map/S03-llm-routing-profiles.md b/.agents/skills/verify-sdk/references/feature-map/S03-llm-routing-profiles.md index 1091297780e..132f2930b05 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S03-llm-routing-profiles.md +++ b/.agents/skills/verify-sdk/references/feature-map/S03-llm-routing-profiles.md @@ -75,7 +75,7 @@ Every model in the scripted bullets is a real `LLM` answering from its own local --check first_run.secondary_in_registry eq true --check first_run.usage_ids contains secondary \ --check first_run.secondary_cost eq 0.14 --check first_run.combined_cost eq 0.42 ``` - Known failure (reproduced 2026-10-13, still failing 2026-11-09, [#5639](https://github.com/OpenHands/software-agent-sdk/issues/5639)): an LLM added to `conversation.llm_registry` before the first `send_message` never reaches `conversation_stats`. The conversation subscribes the stats to its registry only when it initializes the agent (on the first message), and `subscribe` does not replay LLMs already registered, so `secondary` sits in the registry while `usage_to_metrics` has only `agent`, and its 0.14 is missing from the combined cost (0.28). + Known failure (reproduced 2026-11-12, [#5639](https://github.com/OpenHands/software-agent-sdk/issues/5639)): an LLM added to `conversation.llm_registry` before the first `send_message` never reaches `conversation_stats`. The conversation subscribes the stats to its registry only when it initializes the agent (on the first message), and `subscribe` does not replay LLMs already registered, so `secondary` sits in the registry while `usage_to_metrics` has only `agent`, and its 0.14 is missing from the combined cost (0.28). - **Every LLM the agent reaches is registered (`S03.agent-llm-discovery`).** An agent whose LLM is a `MultimodalRouter` and whose condenser has its own LLM; then an agent and condenser that share a `usage_id`. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S04-local-conversation.md b/.agents/skills/verify-sdk/references/feature-map/S04-local-conversation.md index c4680c9fbe6..4666591dc60 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S04-local-conversation.md +++ b/.agents/skills/verify-sdk/references/feature-map/S04-local-conversation.md @@ -28,7 +28,7 @@ Examples: `examples/01_standalone_sdk/06_interactive_terminal_w_reasoning.py`, ` - `S04.secrets-masking`: secret values echoed by a command are replaced by `` in observations and never reach the persisted events. Unit: `tests/tools/terminal/test_secrets_masking.py`, `tests/cross/test_agent_secrets_integration.py`. Example: `examples/01_standalone_sdk/12_custom_secrets.py`. Real-LLM: none. - `S04.visualizer-default`: by default a `DefaultConversationVisualizer` prints each event as a Rich panel; `visualizer=None` turns it off; a class or instance can be passed. Unit: `tests/sdk/conversation/local/test_conversation_visualize_param.py`. Example: `examples/01_standalone_sdk/10_persistence.py`. Real-LLM: none. - `S04.visualizer-custom`: a `ConversationVisualizerBase` subclass passed as `visualizer=` receives every event through `on_event`; each event type has its own `visualize` rendering. Unit: `tests/sdk/conversation/test_visualizer.py`. Example: `examples/01_standalone_sdk/26_custom_visualizer.py`. Real-LLM: none. -- `S04.fork`: `fork()` deep-copies the conversation under a new id (or `conversation_id=`), starts IDLE, leaves the source untouched, resets metrics unless `reset_metrics=False`, accepts `agent=` and `tags=` (`title=` is deprecated and ignored locally since #4814; only `RemoteConversation` stores it), shares the workspace, and persists so the fork survives a reload. Unit: `tests/sdk/conversation/local/test_fork.py`. Example: `examples/01_standalone_sdk/48_conversation_fork.py`. Real-LLM: none. +- `S04.fork`: `fork()` deep-copies the conversation under a new id (or `conversation_id=`), starts IDLE, leaves the source untouched, resets metrics unless `reset_metrics=False`, accepts `agent=` and `tags=` (`title=` is deprecated and ignored locally; only `RemoteConversation` stores it), shares the workspace, and persists so the fork survives a reload. Unit: `tests/sdk/conversation/local/test_fork.py`. Example: `examples/01_standalone_sdk/48_conversation_fork.py`. Real-LLM: none. - `S04.fork-branch`: events carry parent ids forming a tree; `fork(from_event_id=...)` copies only that branch, and `navigate_to(event_id)` moves HEAD so the next event starts a sibling branch (`None` empties the active branch). Unit: `tests/sdk/conversation/local/test_conversation_tree.py`. Example: `examples/01_standalone_sdk/48_conversation_fork.py`. Real-LLM: none. - `S04.goal-loop`: `run_goal(conversation, objective, judge_llm, max_iterations=N)` sends the objective, runs, asks the judge LLM for a verdict, re-prompts with the missing items, and returns a `GoalOutcome` with status `"complete"` or `"capped"`, the iteration count and the verdict. Unit: `tests/sdk/conversation/goal/`. Example: `examples/01_standalone_sdk/54_goal_completion_loop.py`. Real-LLM: none. - `S04.interactive-terminal`: with `Tool(name=TerminalTool.name, params={"no_change_timeout_seconds": 3})` the agent can start `python3`, feed it input and exit; an `LLMConvertibleEvent` callback collects the raw LLM messages. Unit: `tests/tools/terminal/test_terminal_session.py::test_python_interactive_input`, `::test_interactive_command`. Example: `examples/01_standalone_sdk/06_interactive_terminal_w_reasoning.py`. Real-LLM: `tests/integration/tests/t07_interactive_commands.py`. @@ -80,13 +80,13 @@ Scripted bullets use the SDK's `TestLLM` (`--script`, or `make_llm(script=...)` --check names_match eq true --check numbered_in_order eq true --check files_follow_event_ids eq true \ --check callback_saw_file_every_time eq true --check files_grew_one_by_one eq true --check status eq finished ``` -- **Damaged event files (`S04.persist-events`).** A finished conversation is reopened after one change at a time to its `events/` folder: an empty file (what an interrupted write leaves), invalid JSON, a deleted file. A deleted file is skipped and the other events reload (#6197); an empty or invalid file still makes the reload fail with an error that names the file or the JSON problem (the recovery `S05.event-log-recovery` expects). +- **Damaged event files (`S04.persist-events`).** A finished conversation is reopened after one change at a time to its `events/` folder: an empty file (what an interrupted write leaves), invalid JSON, a deleted file. The conversation reopens every time without an error: a deleted file is skipped and the other events reload; an empty or invalid file is read as a `ConversationErrorEvent` stand-in (code `UnreadableEventFile`) in the same place, so the event count is unchanged (`S05.event-log-recovery`). ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) control-sdk program S04/lifecycle --arg corrupt \ --check events_before eq 6 --check intact.events eq 6 \ - --check empty_middle.error contains "Missing event file" --check empty_last.error contains "Missing event file" \ - --check invalid_json_middle.error contains "Invalid JSON" \ + --check empty_middle.error missing --check empty_middle.events eq 6 --check empty_last.error missing --check empty_last.events eq 6 \ + --check invalid_json_middle.error missing --check invalid_json_middle.events eq 6 \ --check missing_middle.error missing --check missing_middle.events eq 5 \ --check missing_last.error missing --check missing_last.events eq 5 ``` @@ -239,7 +239,7 @@ Scripted bullets use the SDK's `TestLLM` (`--script`, or `make_llm(script=...)` --check repl.collected_tool_messages eq 5 --check repl.collected_roles contains tool \ --check prompt_is_input.waiting eq true --check prompt_is_input.reply contains "hello John 42" ``` - Before #6651, with the default tmux pane pool, every terminal call first interrupted a command that was still running (`_prepare_pooled_session` sent Ctrl+C after a no-change timeout), so a program waiting on `input()` could not be answered (`is_input=True` returned `No previous running command to interact with.`, and a plain `John` reached bash). Follow-up calls now reach the running command's pane untouched. + Known failure (reproduced 2026-11-12, [#6752](https://github.com/OpenHands/software-agent-sdk/issues/6752)): with the default tmux pane pool, an `is_input=True` follow-up reaches the running command untouched (`prompt_is_input` passes), but a plain command typed into the running REPL is preceded by Ctrl+C and a `clear` typed into Python, so `repl.reply` starts with `KeyboardInterrupt` and `NameError: name 'clear' is not defined` and `42` falls past the 200 characters the program keeps. The subprocess backend refuses the plain command instead. - **Final response, title and tags (`S04.final-response`, `S04.title-and-tags`).** `get_agent_final_response` on finished conversations; `generate_title` from the agent's LLM, an explicit LLM, a reply with `` reasoning, a failing LLM and no user message; tags kept on disk, bad keys and long values refused. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S05-events.md b/.agents/skills/verify-sdk/references/feature-map/S05-events.md index 3b282b984de..73188dc3d07 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S05-events.md +++ b/.agents/skills/verify-sdk/references/feature-map/S05-events.md @@ -102,7 +102,7 @@ Preconditions: control-sdk program S05/damaged_log --arg "$CID" --arg delete \ --check reopen_error missing --check lost_beyond_damaged eq 0 ``` - Expected: the conversation reopens with every readable event, and the damaged file is reported once in the log. Before #6649, reopening over a corrupted file failed with a pydantic `ValidationError` ("Invalid JSON"), and over an empty file with `FileNotFoundError: Missing event file`; a deleted file has recovered since #6197. The event log now reads an unreadable file as a `ConversationErrorEvent` stand-in (code `UnreadableEventFile`) with the same id and place, so the event count is unchanged. + Expected: the conversation reopens with every readable event, and the damaged file is reported once in the log. A corrupted, empty or deleted event file does not stop the reopen. The event log reads an unreadable file as a `ConversationErrorEvent` stand-in (code `UnreadableEventFile`) with the same id and place, so the event count is unchanged. - **The same, live (`S05.json-roundtrip`, `S05.events-to-messages`, `S05.openai-format`).** A real model does the task its own way; the reload must hold for whatever it produced. ```sh export SDK_VERIFY_RUN=$(control-sdk new --llm deepseek-flash --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S06-context-condensers.md b/.agents/skills/verify-sdk/references/feature-map/S06-context-condensers.md index f21fa14a8a8..211c8467de3 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S06-context-condensers.md +++ b/.agents/skills/verify-sdk/references/feature-map/S06-context-condensers.md @@ -22,6 +22,7 @@ Examples: `examples/01_standalone_sdk/14_context_condenser.py`, `examples/01_sta - `S06.memory-write-path`: the prompt tells the agent where to write memory, and that path is the same one the loader reads (honoring `OH_PERSISTENCE_DIR`); the resolved memory text is not serialized with the `AgentContext`. Unit: `tests/sdk/context/test_memory.py::test_instructed_write_path_matches_loader_read_path_with_persistence_dir`, `tests/sdk/context/test_agent_context_serialization.py::test_memory_context_excluded_from_serialization`. Example: `examples/01_standalone_sdk/55_persistent_memory.py`. Real-LLM: none. - `S06.view`: the `View` the condenser sees hides forgotten events, inserts the summary at `summary_offset`, keeps several summaries in order, and always drops `CondensationRequest` events. Unit: `tests/sdk/context/view/test_view.py`, `tests/sdk/context/view/test_view_multi_summary.py`, `tests/sdk/context/view/test_view_append_event.py`. Example: none. Real-LLM: none. - `S06.view-atomicity`: forgetting never splits an action from its observation, a parallel tool batch, or a tool loop (needed for Anthropic thinking blocks), and observations stay unique. Unit: `tests/sdk/context/view/properties/`, `tests/sdk/context/view/test_view_batch_atomicity.py`, `tests/sdk/context/view/test_view_condensation_batch_atomicity.py`, `tests/sdk/context/view/test_view_tool_loop_boundaries.py`, `tests/sdk/context/view/test_manipulation_indices.py`. Example: none. Real-LLM: `tests/integration/tests/c01_thinking_block_condenser.py`. +- `S06.observation-dispositions`: `get_observation_dispositions(events)` (from `openhands.sdk.context.view`) returns one `ObservationDisposition(tool_call_id, canonical_event_id, out_of_view_event_ids)` per tool call, in input order, without changing the events: the first observation-like outcome (observation, rejection or `AgentErrorEvent`) is canonical and later ones are out of view, exactly the observations the `View` keeps. A recovery-synthesized `AgentErrorEvent` carries `recovery_reason` (`"server_restart"`, set by the agent server; `None` otherwise), which survives a JSON round trip. Unit: `tests/sdk/context/view/test_observation_disposition.py`, `tests/sdk/event/test_event_serialization.py`. Example: none. Real-LLM: none. - `S06.summarizing-size`: `LLMSummarizingCondenser(max_size=N, keep_first=K)` condenses once the view exceeds N events, keeping the first K events and a tail, and summarizes the forgotten middle (building on the previous summary); `keep_first` must be below `max_size // 2`. Unit: `tests/sdk/context/condenser/test_llm_summarizing_condenser.py`. Example: `examples/01_standalone_sdk/14_context_condenser.py`. Real-LLM: `tests/integration/tests/c05_size_condenser.py`. - `S06.summarizing-tokens`: with `max_tokens` (or the agent LLM's effective input limit, whichever is smaller) the condenser triggers on token count and forgets enough events to get under it. Unit: `tests/sdk/context/condenser/test_llm_summarizing_condenser.py::test_condense_with_token_limit_exceeded`, `::test_token_limit_uses_stricter_configured_or_agent_limit`, `tests/sdk/context/condenser/test_utils.py`. Example: none. Real-LLM: `tests/integration/tests/c04_token_condenser.py`. - `S06.summarizing-reasons`: request, event-count and token reasons combine and the most aggressive condensation wins; `minimum_progress` (default 0.1) rejects condensations that forget too little. Unit: `tests/sdk/context/condenser/test_llm_summarizing_condenser.py::test_most_aggressive_condensation_chosen`, `::test_minimum_progress_threshold_not_met`. Example: none. Real-LLM: none. @@ -141,6 +142,16 @@ The programs under `programs/S06/` record every scripted LLM call (`recorder.py` --check keep_first_too_big_error eq ValidationError --check noop_same_view eq true ``` Each reason forgets more: the request alone halves the view, the size reason forgets down to `max_size // 2`, and adding a token cap forgets the most; the combination takes the most aggressive. +- **Observation dispositions (`S06.observation-dispositions`).** The events of a finished scripted conversation (a parallel pair of terminal calls, one more call, `finish`), with an `AgentErrorEvent(recovery_reason="server_restart")` for the first call inserted ahead of that call's real observation, as a restarted agent server persists it, and reloaded with `Event.model_validate_json`. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --field run) + control-sdk program S06/condense --arg dispositions \ + --check recovery_reason_roundtrip eq server_restart --check recovery_reason_default eq null \ + --check tool_calls eq 4 --check dispositions eq 4 --check order_follows_input eq true \ + --check first_canonical_is_recovery eq true --check first_out_of_view_is_real eq true --check others_single eq true \ + --check view_observations_match eq true --check events_unchanged eq true --check actions_only eq 0 + ``` + The recovery outcome, being first, is canonical and the late real result is out of view; the `View` keeps exactly the canonical outcomes, and actions alone have no dispositions. - **Soft versus hard requirements (`S06.soft-vs-hard`).** With `minimum_progress=0.95` nothing is worth forgetting: size pressure leaves the run alone (no summary call), while `conversation.condense()` (a hard request) resets the context. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S07-skills.md b/.agents/skills/verify-sdk/references/feature-map/S07-skills.md index 5cd189979c9..c9aaaf5ad8b 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S07-skills.md +++ b/.agents/skills/verify-sdk/references/feature-map/S07-skills.md @@ -100,7 +100,7 @@ Each run has its own `HOME` and `OH_PERSISTENCE_DIR`, so user skills, the instal --check longdesc_full_in_prompt eq false --check longdesc_points_to_invoke_skill eq true \ --check longdesc_leaks_location eq false ``` - Expected: the description is cut at 1024 characters with `Call invoke_skill(name="longdesc") to load the full skill`, and no file location is shown (`to_prompt` omits `` on purpose). `Skill` truncates a long description already when the `SKILL.md` is loaded; since #6644 that notice also points at `invoke_skill` and not at the file. + Expected: the description is cut at 1024 characters with `Call invoke_skill(name="longdesc") to load the full skill`, and no file location is shown (`to_prompt` omits `` on purpose). `Skill` truncates a long description already when the `SKILL.md` is loaded, and that notice points at `invoke_skill`, not at the file. - **Path rules (`S07.path-rules`).** A `paths: [src/**/*.py]` AgentSkills rule and a nested `docs/AGENTS.md`; the scripted agent views `README.md`, `src/app/main.py`, `src/app/other.py` and `docs/guide.md` with `file_editor`. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -185,7 +185,7 @@ Each run has its own `HOME` and `OH_PERSISTENCE_DIR`, so user skills, the instal export SDK_VERIFY_RUN=$(control-sdk new --field run) control-sdk exec --expect-output "DEMO 1: Installing local skills" -- python examples/05_skills_and_plugins/03_managing_installed_skills/main.py ``` - Known failure (reproduced 2026-11-03, not yet filed): the example stops in DEMO 3 with `KeyError: 'skills'`. It reads `.installed.json` as `metadata["skills"][name]`, but the install store now records every kind under `"extensions"` (`InstallationMetadata` in `openhands-sdk/openhands/sdk/extensions/installation/metadata.py` reads `skills` and `plugins` only as legacy keys). The SDK calls themselves work; the example's own asserts are stale, and no CI job runs it (see the gotcha below). + Known failure (reproduced 2026-11-12, [#6248](https://github.com/OpenHands/software-agent-sdk/issues/6248)): the example stops in DEMO 3 with `KeyError: 'skills'`. It reads `.installed.json` as `metadata["skills"][name]`, but the install store now records every kind under `"extensions"` (`InstallationMetadata` in `openhands-sdk/openhands/sdk/extensions/installation/metadata.py` reads `skills` and `plugins` only as legacy keys). The SDK calls themselves work; the example's own asserts are stale, and no CI job runs it (see the gotcha below). - **Mixed marketplace example (`S07.marketplace-install`).** Network; `--install` writes to the run's install store (its `HOME` and `OH_PERSISTENCE_DIR` are private). ```sh export SDK_VERIFY_RUN=$(control-sdk new --llm deepseek-flash --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S08-plugins-marketplace.md b/.agents/skills/verify-sdk/references/feature-map/S08-plugins-marketplace.md index 2df10f6f5d6..909c0dc60e3 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S08-plugins-marketplace.md +++ b/.agents/skills/verify-sdk/references/feature-map/S08-plugins-marketplace.md @@ -24,7 +24,7 @@ Examples: `examples/05_skills_and_plugins/02_loading_plugins/main.py`, `examples - `S08.source-parsing`: `github:owner/repo`, https, ssh and `git://` URLs (GitHub, GitLab, Bitbucket, self-hosted) and local paths are classified; GitHub `blob`/`tree` URLs split into repo, branch and subpath; `repo_path` must be relative without `..`. Unit: `tests/sdk/extensions/test_fetch.py`, `tests/sdk/plugin/test_source.py`, `tests/sdk/plugin/test_plugin_loader.py::TestPluginSource`. Example: none. Real-LLM: none. - `S08.installed-plugins`: `install_plugin(source, ref=, installed_dir=, force=)` copies a fetched plugin into `~/.openhands/plugins/installed/` and records it in `.installed.json`; `list_installed_plugins`, `load_installed_plugins` (enabled only), `disable_plugin`/`enable_plugin`, `get_installed_plugin`, `update_plugin` and `uninstall_plugin` act on that store. Unit: `tests/sdk/plugin/test_installed_plugins.py`. Example: `examples/05_skills_and_plugins/02_loading_plugins/main.py` (demos 2 to 6). Real-LLM: none. - `S08.installation-manager`: the generic `InstallationManager[T]` validates names (no path traversal), records requested and resolved refs, keeps the enabled flag across `force` reinstalls, re-clones on `update`, and migrates legacy `plugins`/`skills` keys of `.installed.json` into `extensions`. Unit: `tests/sdk/extensions/installation/`. Example: none. Real-LLM: none. -- `S08.ambient-plugins`: `load_user_plugins()` (`~/.agents/plugins`, then `~/.openhands/plugins`, then enabled installed plugins) and `load_project_plugins(work_dir)` (work dir and git root, `.agents/plugins` then `.openhands/plugins`) are merged into every `LocalConversation`, project over user; an explicitly attached plugin of the same name wins, and ambient plugins are not listed in `resolved_plugins`. Unit: `tests/sdk/plugin/test_discovery.py`, `tests/sdk/conversation/test_local_conversation_plugins.py::TestAmbientPluginAutoLoad`. Example: none. Real-LLM: none. +- `S08.ambient-plugins`: `load_user_plugins()` (`~/.agents/plugins`, then `~/.openhands/plugins`, then enabled installed plugins) and `load_project_plugins(work_dir)` (work dir and git root, `.agents/plugins` then `.openhands/plugins`) are merged into every `LocalConversation`, project over user; an explicitly attached plugin of the same name wins, and ambient plugins are not listed in `resolved_plugins`. The conversation state records the skill and MCP server names ambient plugins contributed (`ambient_plugin_skills`, `ambient_plugin_mcp_servers`), so a resumed conversation drops a since-disabled or removed plugin's skills and MCP servers before re-discovery; a failed discovery keeps the previous contributions. Unit: `tests/sdk/plugin/test_discovery.py`, `tests/sdk/conversation/test_local_conversation_plugins.py::TestAmbientPluginAutoLoad`. Example: none. Real-LLM: none. - `S08.marketplace-manifest`: `Marketplace.load(dir)` reads `.plugin/marketplace.json` or `.claude-plugin/marketplace.json` with `name`, `owner`, `metadata`, `plugins` (string source or `{source: github|url, repo, url, ref, path}`) and `skills`, and `resolve_plugin_source(entry)` turns an entry into `(source, ref, repo_path)` relative to the marketplace. Unit: `tests/sdk/marketplace/test_marketplace.py`. Example: `examples/05_skills_and_plugins/04_mixed_marketplace_skills/main.py` (default mode prints the manifest). Real-LLM: none. - `S08.marketplace-registry`: `MarketplaceRegistry(registrations)` rejects duplicate names, fetches and caches each manifest lazily (keeping the resolved ref), and `resolve_plugin("name@market")` or `resolve_plugin("name")` returns a `PluginSource`, raising `PluginNotFoundError`, `AmbiguousPluginError` or `MarketplaceNotFoundError`. Unit: `tests/sdk/marketplace/test_marketplace_registry.py`. Example: none. Real-LLM: none. - `S08.marketplace-auto-load`: `AgentContext(registered_marketplaces=[MarketplaceRegistration(name, source, ref, repo_path, auto_load=True | [names])])` loads the chosen plugins and standalone skills when the conversation starts; a plugin wins over a standalone skill of the same name, explicit `plugins=` win over auto-loaded ones, a marketplace that fails to fetch is skipped, and `auto_load=False` only registers it. Unit: `tests/sdk/conversation/test_local_conversation_plugins.py::TestLocalConversationPlugins` (`test_auto_load_marketplace_plugins`, `test_auto_load_marketplace_standalone_skills`, `test_plugin_wins_over_standalone_on_name_collision`, `test_auto_load_marketplace_continues_after_fetch_failure`, `test_registered_only_marketplace_does_not_auto_load`, `test_explicit_plugins_override_auto_loaded_marketplace_plugins`), `tests/sdk/skills/test_installed_skills.py::TestLoadMarketplaceStandaloneSkills`. Example: none. Real-LLM: none. @@ -147,7 +147,7 @@ The programs build every plugin, marketplace and git repository inside the run ( - The git source is fetched through the secret. - `TOKEN` comes from the conversation secret (`nekot-terces`), never from the host. - Known failure (reproduced 2026-10-13, not yet filed): the server receives the host's `TOKEN` (`nekot-tsoh` is `host-token` reversed) instead of the conversation secret `secret-token`. A placeholder with no host variable (`${PLUGIN_KEY}`) correctly takes the secret. The cause is that `Plugin.load` reads `.mcp.json` through `load_mcp_config` with no secret lookup. `expand_mcp_variables` uses `check_env=True`, so at load time `${TOKEN}` is replaced from `os.environ`. When the conversation later applies its secrets, the placeholder is already gone. Any variable a plugin names that also exists in the agent process's environment leaks that value into the plugin's MCP server, and the user's secret is ignored. + Known failure (reproduced 2026-11-12, [#6643](https://github.com/OpenHands/software-agent-sdk/issues/6643)): the server receives the host's `TOKEN` (`nekot-tsoh` is `host-token` reversed) instead of the conversation secret `secret-token`. A placeholder with no host variable (`${PLUGIN_KEY}`) correctly takes the secret. The cause is that `Plugin.load` reads `.mcp.json` through `load_mcp_config` with no secret lookup. `expand_mcp_variables` uses `check_env=True`, so at load time `${TOKEN}` is replaced from `os.environ`. When the conversation later applies its secrets, the placeholder is already gone. Any variable a plugin names that also exists in the agent process's environment leaks that value into the plugin's MCP server, and the user's secret is ignored. - **A plugin source that cannot be fetched (`S08.conversation-attach`, `S08.fetch-cache`).** ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -175,7 +175,7 @@ The programs build every plugin, marketplace and git repository inside the run ( control-sdk program S08/attach --arg fetch-ref \ --check default_version eq v2 --check v1_after_cached_clone eq v1 --check v1_resolved_is_v1_sha eq true ``` - Expected: the second fetch checks out the tag and returns v1 with v1's SHA. Fixed: a known failure on 2026-10-13 (the shallow single-branch cache silently returned v2 content and SHA); it passes since #5685, re-verified 2026-11-03. + Expected: the second fetch checks out the tag and returns v1 with v1's SHA. - **The install store (`S08.installed-plugins`, `S08.ambient-plugins`).** Plugins are installed into the default store, which lives under the run's `OH_PERSISTENCE_DIR`: - a git plugin at tag `v1` - a local plugin @@ -196,6 +196,18 @@ The programs build every plugin, marketplace and git repository inside the run ( --check uninstall eq true --check uninstall_again eq false --check dir_removed eq true \ --check listed_end eq '["localplug"]' --check get_missing eq null ``` +- **Resuming after an ambient plugin is disabled (`S08.ambient-plugins`).** An installed Claude Code plugin `resplug` (a skill, a command and a `.mcp.json` server) loads into a persisted conversation on its first message. The plugin is then disabled, the same `conversation_id` is reopened with its persisted agent (`LocalConversation(agent=None, conversation_id=...)`) and sent a message; then it is enabled and reopened again. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --field run) + control-sdk program S08/store --arg resume \ + --check enabled.skills eq '["resplug-skill", "resplug:review"]' --check enabled.mcp_servers eq '["resplug-mcp"]' \ + --check enabled.recorded_skills eq '["resplug-skill", "resplug:review"]' --check enabled.recorded_mcp eq '["resplug-mcp"]' \ + --check disable eq true --check resumed_same_id eq true \ + --check after_disable.skills eq '[]' --check after_disable.mcp_servers eq '[]' \ + --check after_disable.recorded_skills eq '[]' --check after_disable.recorded_mcp eq '[]' \ + --check after_enable.skills eq '["resplug-skill", "resplug:review"]' --check after_enable.mcp_servers eq '["resplug-mcp"]' + ``` + The resumed conversation's persisted agent no longer carries the disabled plugin's skill, command or MCP server, and its ownership record is empty; enabling the plugin brings them back on the next resume. - **Updating a plugin that was installed at a tag (`S08.installed-plugins`).** This is the same program. `gitplug` was installed at `v1` and `main` has since moved to 2.0.0. `update_plugin("gitplug")` then runs; it re-fetches with `ref=None`. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -203,7 +215,7 @@ The programs build every plugin, marketplace and git repository inside the run ( --check update_requested_ref eq null --check update_version eq 2.0.0 \ --check update_resolved_is_v2 eq true --check installed_file_version eq 2.0.0 ``` - Expected: the plugin moves to the latest version, as the docstring says ("Update an installed extension to the latest version"). Fixed: a known failure on 2026-10-13 (the plugin stayed at 1.0.0 while `requested_ref` was cleared, the same cache as the previous bullet); it passes since #5685, re-verified 2026-11-03. + Expected: the plugin moves to the latest version, as the docstring says ("Update an installed extension to the latest version"). - **The generic installation manager and legacy metadata (`S08.installation-manager`).** These calls go to `InstallationManager` directly: - a manifest named `../escaped` - a non-kebab-case name diff --git a/.agents/skills/verify-sdk/references/feature-map/S09-mcp.md b/.agents/skills/verify-sdk/references/feature-map/S09-mcp.md index 26a3b417e3a..1d5ae1f1189 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S09-mcp.md +++ b/.agents/skills/verify-sdk/references/feature-map/S09-mcp.md @@ -147,7 +147,7 @@ Every server here is `programs/S09/mcp_server.py`, a small FastMCP server the pr --check http_after_restart_again.1 eq false --check http_after_restart_again.0 contains count=1 \ --check stdio_after_exit_again.1 eq false --check stdio_after_exit_again.0 contains count=1 ``` - Expected: at most one failed call, then the client reconnects (a new session or process, so the counter starts again). Known failure (reproduced 2026-10-13 and again 2026-11-08 with the `mcp` 2 client, not yet filed): neither recovers. After the restart every call returns `Error calling MCP tool counter: Session not found` (`Session terminated` before #6540); after the stdio server exits, every call returns `Error calling MCP tool counter: Connection closed` (empty before), so the server's tools stay broken for the rest of the conversation. `MCPToolExecutor.call_tool` reconnects only when `client.is_connected()` is false, which stays true in both cases. + Expected: at most one failed call, then the client reconnects (a new session or process, so the counter starts again). Known failure (reproduced 2026-11-12, [#6647](https://github.com/OpenHands/software-agent-sdk/issues/6647)): neither recovers. After the restart every call returns `Error calling MCP tool counter: Session not found`; after the stdio server exits, every call returns `Error calling MCP tool counter: Connection closed`, so the server's tools stay broken for the rest of the conversation. `MCPToolExecutor.call_tool` reconnects only when `client.is_connected()` is false, which stays true in both cases. - **Config model, secrets, schemas, arguments and serialization without an agent (`S09.config-model`, `S09.config-secrets`, `S09.schema-conversion`, `S09.argument-validation`, `S09.observation`, `S09.tool-serialization`).** ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S10-hooks.md b/.agents/skills/verify-sdk/references/feature-map/S10-hooks.md index ca52145375f..81fde715bb4 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S10-hooks.md +++ b/.agents/skills/verify-sdk/references/feature-map/S10-hooks.md @@ -143,7 +143,7 @@ Preconditions: control-sdk exec --model-env --expect-output "Example Complete!" --expect-output "EXAMPLE_COST" -- python examples/01_standalone_sdk/51_agent_hooks/main.py ``` Demo 1 prints `[hook] PreToolUse DENY -> agent-hook:security-reviewer` for the `awk '{print}' /etc/passwd` call; demo 2 prints `Stop` lines from `agent-hook:quality-reviewer` and a `REPORT.md` preview. - Before #6659, demo 2 asked for a `REPORT.md` "describing this repository" in an empty temporary workspace, and DeepSeek Flash went looking for a repository elsewhere on the machine until the 10-iteration cap raised `RuntimeError`. Now demo 2 seeds the workspace with a small project to describe; three runs with DeepSeek Flash on 2026-11-09 completed. + Demo 2 seeds its temporary workspace with a small project and asks for a `REPORT.md` describing it. - **Prompt hooks with a real model, example 57 (`S10.prompt-hook`).** Live; the script asserts both verdicts, so a model that disagrees makes it exit non-zero. ```sh export SDK_VERIFY_RUN=$(control-sdk new --llm deepseek-flash --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S11-subagents-delegation.md b/.agents/skills/verify-sdk/references/feature-map/S11-subagents-delegation.md index 135b54ed5d3..1e63beff98d 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S11-subagents-delegation.md +++ b/.agents/skills/verify-sdk/references/feature-map/S11-subagents-delegation.md @@ -29,6 +29,7 @@ Examples: `examples/01_standalone_sdk/25_agent_delegation.py`, `examples/01_stan - `S11.task-persistence`: with a persisting parent, sub-conversations are stored under `/subagents/` and kept on close; without one they go to a temporary `openhands_tasks_*` dir that `close()` deletes. Unit: `tests/tools/task/test_task_manager.py::TestTaskManagerPersistence`. Example: none. Real-LLM: none. - `S11.task-hooks`: a definition's `hooks` become the sub-conversation's `hook_config`, on create and on resume. Unit: `tests/tools/task/test_task_manager.py::TestTaskManagerHooks`. Example: none. Real-LLM: none. - `S11.task-observability`: sub-conversations share the parent's prompt cache key and carry delegate trace metadata (`is_delegate`, `task_id`, `subagent_type`, `parent_session_id`, parent span link) and the `delegate` tag. Unit: `tests/tools/task/test_task_manager.py::TestTaskManager::test_sub_agents_inherit_parent_prompt_cache_key`, `::test_marks_conversation_as_delegate_with_linking_metadata`, `::test_delegate_metadata_includes_parent_span_link`. Example: none. Real-LLM: none. +- `S11.task-secrets`: secrets the parent conversation holds (`conversation.update_secrets(...)`) resolve in a task sub-agent's terminal, on create and on resume, and stay masked in what comes back. Unit: **none (untested)**. Example: none. Real-LLM: none. - `S11.task-confirmation`: a definition's `permission_mode` sets the sub-conversation's confirmation policy, omission inherits the parent's; when the sub-agent waits for confirmation, `TaskToolSet.create(confirmation_handler=...)` decides, and without a handler the actions are approved. Unit: **none (untested)**. Example: none. Real-LLM: none. - `S11.sub-agent-scope`: `SubAgentScope(tools=True, mcp_servers=True)` (the `sub_agent_scope` tool param) hides and refuses sub-agents whose tools or MCP servers the parent lacks, compares built-in tool names canonically, propagates to nested delegation tools, and says so when no sub-agent fits. Unit: `tests/sdk/subagent/test_subagent_scope.py`, `tests/tools/task/test_sub_agent_scope.py`. Example: none. Real-LLM: none. - `S11.delegate-executor`: the older `DelegateExecutor` (`spawn` named sub-agents, then `delegate` a dict of tasks run in parallel threads) closes replaced and spawned sub-agents, rolls back a partial spawn, merges metrics, and inherits the parent's persistence dir. Unit: `tests/tools/delegate/test_delegation.py`. Example: none. Real-LLM: none. @@ -79,7 +80,16 @@ Programs give each programmatic sub-agent its own scripted LLM (`programs/S11/s1 --check sub_cache_key_is_parent.0 eq true --check sub_cache_key_is_parent.1 eq true \ --check sub_cache_key_is_parent.2 eq true --check sub_session_differs eq true ``` - Expected: every sub-agent call carries the parent conversation's id as `prompt_cache_key`, on create and on resume. Known failure (reproduced 2026-10-13, not yet filed): the two calls of the created task carry the parent's key, but the call made after `task(resume=...)` carries the sub-conversation's own id (`sub_cache_key_is_parent` is `[true, true, false]`): `TaskManager._resume_task` builds the `LocalConversation` without `prompt_cache_key=...`, unlike `_get_conversation`. + Expected: every sub-agent call carries the parent conversation's id as `prompt_cache_key`, on create and on resume. Known failure (reproduced 2026-11-12, [#6655](https://github.com/OpenHands/software-agent-sdk/issues/6655)): the two calls of the created task carry the parent's key, but the call made after `task(resume=...)` carries the sub-conversation's own id (`sub_cache_key_is_parent` is `[true, true, false]`): `TaskManager._resume_task` builds the `LocalConversation` without `prompt_cache_key=...`, unlike `_get_conversation`. +- **Delegates see the parent's secrets (`S11.task-secrets`).** A persisted parent registers `PROBE_TOKEN` with `update_secrets`, runs `echo probe=${PROBE_TOKEN:+set} value=$PROBE_TOKEN` itself, then delegates the same command to the programmatic sub-agent `prober` and resumes that task for a second run. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --field run) + control-sdk program S11/task_secrets \ + --check execution_status eq finished --check parent_probe eq set \ + --check task_statuses len-eq 2 --check parent_masked eq true --check secret_leaked eq false \ + --check create_probe eq set --check resume_probe eq set + ``` + Expected: the secret resolves in the sub-agent's shell, on create and on resume, as it does in the parent's (`create_probe` and `resume_probe` are `set`), and the parent's output stays masked. - **Delegate trace metadata (`S11.task-observability`), blocked.** `is_delegate`, `task_id`, `subagent_type`, `parent_session_id`, the parent span link and the `delegate` tag go only to the observability backend; proving them needs a Laminar/OpenTelemetry collector to read spans from. - **Registry, Markdown definitions and factories (`S11.register-programmatic`, `S11.register-if-absent`, `S11.markdown-schema`, `S11.file-discovery`, `S11.file-precedence`, `S11.plugin-agents`, `S11.definition-factory`, `S11.definition-model`).** The program lays out agent files in the workspace (`.agents/agents/`, `.openhands/agents/`, a `README.md`, a nested dir), the user's home (`~/.agents/agents/`) and `$OH_PERSISTENCE_DIR/agents/`, saves an LLM profile, and goes through `openhands.sdk.subagent`. ```sh @@ -125,7 +135,7 @@ Programs give each programmatic sub-agent its own scripted LLM (`programs/S11/s1 control-sdk program S11/builtins \ --check example_in_description.general-purpose eq true --check example_in_description.web-researcher eq true ``` - Expected: with all four built-ins registered, the description holds an example for each. Before #6656, `TASK_TOOL_EXAMPLES` keyed two of them as `"general purpose"` and `"web researcher"` (spaces), so only the `code-explorer` and `bash-runner` examples appeared; now the keys match the registered names. + Expected: with all four built-ins registered, the description holds an example for each, keyed by the names the built-ins register under. - **Workspace Markdown agents picked up by the conversation (`S11.conversation-autoregister`, `S11.builtin-agents`, `S11.task-persistence`).** The workspace holds `.agents/agents/ws-reviewer.md`; the program never calls `register_file_agents`. A non-persisted parent delegates to it and to the built-in `bash-runner`. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S13-security-confirmation.md b/.agents/skills/verify-sdk/references/feature-map/S13-security-confirmation.md index f05dad491fd..2d6eb626195 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S13-security-confirmation.md +++ b/.agents/skills/verify-sdk/references/feature-map/S13-security-confirmation.md @@ -180,7 +180,7 @@ Preconditions: --check async.tool_step.messages.0 eq "the values are and " \ --check async.tool_step.seconds lt 10 ``` - Expected: both cases are masked in well under a second. Before #6642 the tool case made the lookup inside the tool, while `arun()` held the state lock, so the server's `update_secrets` call waited on that lock and the value stayed unmasked after httpx's 30 s timeout; `astep` now resolves lookups with the lock released before running tools, as it does before masking an agent message. + Expected: both cases are masked in well under a second. `astep` resolves lookups with the state lock released before running tools, as it does before masking an agent message, so a server's `update_secrets` call never waits on that lock. - **A real model labels its own actions (`S13.llm-analyzer`, `S13.confirm-wait`, `S13.confirm-approve-reject`).** Live. The model is asked to run `ls` and then `rm -rf scratch`; `ConfirmRisky()` stops, and the program rejects until the agent finishes. A model that declines to propose the deletion at all (DeepSeek Flash did on 2026-11-03: it only listed and read `scratch/`) leaves the confirmation path unexercised, and the program exits 3. ```sh export SDK_VERIFY_RUN=$(control-sdk new --llm deepseek-flash --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S14-core-tools.md b/.agents/skills/verify-sdk/references/feature-map/S14-core-tools.md index 81f71830da9..e1442fb0c2f 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S14-core-tools.md +++ b/.agents/skills/verify-sdk/references/feature-map/S14-core-tools.md @@ -132,7 +132,7 @@ Preconditions: control-sdk program S14/terminal --arg auto \ --check escape_filtered eq true --check dsr_reply_visible eq false --check after_dsr_clean eq true ``` - Expected: the query is dropped and nothing else shows, as on the subprocess backend. Known failure (reproduced 2026-10-13, still failing 2026-11-09, [#6652](https://github.com/OpenHands/software-agent-sdk/issues/6652)): the query itself is filtered, but tmux answers it into the shell's input; the observation shows `^[[13;2R` and the next command's output starts with the echoed earlier command and `bash: syntax error near unexpected token ';'`. + Expected: the query is dropped and nothing else shows, as on the subprocess backend. Known failure (reproduced 2026-11-12, [#6652](https://github.com/OpenHands/software-agent-sdk/issues/6652)): the query itself is filtered, but tmux answers it into the shell's input; the observation shows `^[[13;2R` and the next command's output starts with the echoed earlier command and `bash: syntax error near unexpected token ';'`. - **A reset keeps the configured backend (`S14.terminal-reset`, `S14.terminal-backend-select`).** The subprocess run's `echo shell=$0` after `reset=True`, and the session type at the end. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -140,7 +140,7 @@ Preconditions: --check steps.shell.text contains my-shell \ --check steps.shell_after_reset.text contains my-shell --check backend_at_end eq SubprocessTerminal ``` - Expected: a reset starts a fresh shell of the same kind. Known failure (reproduced 2026-10-13, still failing 2026-11-09, [#6653](https://github.com/OpenHands/software-agent-sdk/issues/6653)): `TerminalExecutor._reset_single_session` builds the new session with `terminal_type=None`, so an executor forced to `subprocess` comes back from a reset on tmux (`backend_at_end` is `TmuxTerminal`, `$0` is `/bin/bash`), dropping `shell_path`. + Expected: a reset starts a fresh shell of the same kind. Known failure (reproduced 2026-11-12, [#6653](https://github.com/OpenHands/software-agent-sdk/issues/6653)): `TerminalExecutor._reset_single_session` builds the new session with `terminal_type=None`, so an executor forced to `subprocess` comes back from a reset on tmux (`backend_at_end` is `TmuxTerminal`, `$0` is `/bin/bash`), dropping `shell_path`. - **Windows PowerShell backend (`S14.terminal-windows`), blocked.** Needs a Windows host; on Linux `terminal_type="powershell"` has no PowerShell to start. - **The file editor over a small repository (`S14.file-editor-view`, `S14.file-editor-edit`, `S14.file-editor-validation`, `S14.tool-output-masking`).** The program lays out a nested tree, a hidden file, a PNG, an ASCII PDF, a binary blob, an 11 MB log, a Latin-1 file and a file holding a registered secret; a scripted agent views and edits them. ```sh @@ -186,7 +186,7 @@ Preconditions: --check same_files eq true --check rg.glob_in_path.count eq 100 --check same_truncated eq true \ --check rg.glob_in_path.truncated eq false ``` - Expected: both backends report 100 files and `truncated` false. Fixed: a known failure on 2026-10-13 (ripgrep `glob` reported `truncated: true` for exactly 100 matches); it passes since #6342, re-verified 2026-11-03. + Expected: both backends report 100 files and `truncated` false. - **Tool presets build working agents (`S14.preset-default`, `S14.preset-gpt5-gemini`, `S14.gemini-file-tools`, `S14.apply-patch`).** Each preset's agent does a scripted task with its own file tools; the default preset also reports the tool lists of all three and the SDK's `default_tool_specs()`, and builds a structured finish tool. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -256,4 +256,4 @@ Preconditions: - `glob` with the Python fallback uses a process-wide `os.chdir`, so it declares itself not parallel-safe; with ripgrep it is. - The subprocess backend leaks bracketed-paste switches (`\x1b[?2004l`, `\x1b[?2004h`) into observation text; the tmux backend does not. - `TerminalObservation.timeout` stays `false` even when a hard `timeout` fires; read `exit_code` (-1) and the "[The command timed out after ...]" note instead. -- With the tmux pane pool, checking a pane out interrupts whatever is still running in it, so a command left running by a no-change timeout cannot be continued with `""`, `is_input` or `C-c` (see the known failure in Driving it). +- With the tmux pane pool, a follow-up (`is_input`, an empty command or a `C-x` key) reaches a command left running by a no-change timeout, but a plain command does not: the pool sends that command Ctrl+C and types `clear` before running the new command, so a REPL that ignores Ctrl+C sees `KeyboardInterrupt` and `NameError: name 'clear' is not defined` ([#6752](https://github.com/OpenHands/software-agent-sdk/issues/6752), open, waiting on a decision of which behavior is intended; `S04.interactive-terminal` carries the known failure). The subprocess backend refuses the plain command instead and tells the model to use `is_input`. diff --git a/.agents/skills/verify-sdk/references/feature-map/S15-browser.md b/.agents/skills/verify-sdk/references/feature-map/S15-browser.md index 0698e8b7fdb..8a95b044b9e 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S15-browser.md +++ b/.agents/skills/verify-sdk/references/feature-map/S15-browser.md @@ -13,7 +13,7 @@ Examples: `examples/01_standalone_sdk/15_browser_use.py`, `examples/01_standalon - `S15.toolset-create`: `BrowserToolSet.create(conv_state)` returns the 14 browser tools, all bound to one executor, each with the browser prompt guidance and an MCP-convertible schema; the set takes no LLM-facing parameters and is registered on import. Unit: `tests/tools/browser_use/test_browser_toolset.py` (`test_browser_toolset_create_includes_all_browser_tools`, `test_browser_toolset_create_tools_have_shared_executor`, `test_browser_toolset_create_tools_can_generate_mcp_schema`), `tests/cross/test_automatic_registration.py::test_browser_tool_automatic_registration`. Example: `examples/01_standalone_sdk/15_browser_use.py`. Real-LLM: `tests/integration/tests/t05_simple_browsing.py`. - `S15.shared-executor`: repeated `create()` calls (parent and sub-agents) reuse one executor; construction is serialized without holding the shared lock; a second call's executor config is ignored with a warning; a closing executor is waited for before a new one is built. Unit: `tests/tools/browser_use/test_browser_toolset.py` (`test_browser_toolset_create_multiple_calls_share_executor`, `test_browser_toolset_shared_executor_survives_multiple_subagents`, `test_browser_toolset_warns_when_config_ignored`, `test_browser_toolset_create_waits_for_shared_executor_close`). Example: none. Real-LLM: none. - `S15.degrade-on-failure`: when the executor fails to start, `create()` logs a warning and returns no tools, tool resolution for the rest of the agent succeeds, and the system prompt drops the browser guidance; a profile pinned to the browser still resolves on a browserless runtime. Unit: `tests/tools/browser_use/test_browser_toolset.py` (`test_browser_toolset_create_degrades_when_executor_fails`, `test_resolve_tool_survives_browser_executor_failure`, `test_no_browser_guidance_when_browser_fails_to_start`, `test_migrated_profile_with_pinned_browser_resolves_on_browserless_runtime`). Example: none. Real-LLM: none. -- `S15.chromium-detection`: `BrowserToolExecutor.check_chromium_available()` (cached) finds `google-chrome`, `chrome`, `chromium`, `chromium-browser` or `microsoft-edge` on `PATH`, standard Linux, macOS and Windows install paths (Edge on Windows), or Playwright's browsers (`$PLAYWRIGHT_BROWSERS_PATH` since #5908, then the HOME cache), in that order after the system paths, and returns `None` otherwise; `BrowserToolSet.is_usable()` and `list_usable_tools()` follow it; a missing binary raises with install instructions (`uvx playwright install chromium`). Unit: `tests/tools/browser_use/test_chromium_detection.py`, `tests/cross/test_automatic_registration.py::test_browser_tool_usable_listing_respects_chromium_availability`. Example: none. Real-LLM: none. +- `S15.chromium-detection`: `BrowserToolExecutor.check_chromium_available()` (cached) finds `google-chrome`, `chrome`, `chromium`, `chromium-browser` or `microsoft-edge` on `PATH`, standard Linux, macOS and Windows install paths (Edge on Windows), or Playwright's browsers (`$PLAYWRIGHT_BROWSERS_PATH`, then the HOME cache), in that order after the system paths, and returns `None` otherwise; `BrowserToolSet.is_usable()` and `list_usable_tools()` follow it; a missing binary raises with install instructions (`uvx playwright install chromium`). Unit: `tests/tools/browser_use/test_chromium_detection.py`, `tests/cross/test_automatic_registration.py::test_browser_tool_usable_listing_respects_chromium_availability`. Example: none. Real-LLM: none. - `S15.executor-config`: `headless`, `allowed_domains`, `session_timeout_minutes`, `init_timeout_seconds`, `action_timeout_seconds` (must be positive), `inject_scripts` and extra keyword arguments reach the browser-use server; each executor gets its own profile directory unless `user_data_dir` is given; initialization is lazy and idempotent and times out cleanly. Unit: `tests/tools/browser_use/test_browser_executor.py` (`test_browser_executor_config_passing`, `test_browser_executor_rejects_non_positive_action_timeout`, `test_browser_executor_initialization_lazy`, `test_browser_executor_initialization_idempotent`), `tests/tools/browser_use/test_browser_initialization.py`. Example: none. Real-LLM: none. - `S15.action-routing`: each action type is routed to its server method, an unsupported action and a raised exception come back as error observations, and a slow action is cut off at `action_timeout_seconds`. Unit: `tests/tools/browser_use/test_browser_executor.py` (`test_browser_executor_action_routing_*`, `test_browser_executor_unsupported_action_handling`, `test_browser_executor_error_wrapping`, `test_browser_executor_timeout_wrapping_live_service`). Example: none. Real-LLM: none. - `S15.session-recovery`: consecutive timeouts reset the browser session after the third; the last try before a reset uses a fixed 30-second timeout (`DEGRADED_TIMEOUT_SECONDS`, shorter than the 300-second default but longer than a smaller `action_timeout_seconds`); a success resets the counter; ordinary action errors do not count. Unit: `tests/tools/browser_use/test_browser_executor.py` (`test_issue_2412_consecutive_failures_reset_session`, `test_issue_2412_success_resets_failure_counter`, `test_issue_2412_action_errors_do_not_trigger_reset`, `test_issue_2412_degraded_timeout_after_failures`). Example: none. Real-LLM: none. @@ -64,7 +64,6 @@ Preconditions: --check steps.back.text eq "Navigated back" --check steps.state_after_back.text contains index.html \ --check steps.new_tab.text contains "Opened new tab with URL" --check tabs_seen eq 2 \ --check steps.switch_tab.text contains "Switched to tab" --check steps.close_tab.text contains "Closed tab" \ - --check tabs_left eq 1 \ --check storage_has_page_values eq true --check steps.set_storage.text eq "Storage set successfully" \ --check storage_restored eq true \ --check steps.missing_element.text contains "Element with index 99999 not found" \ @@ -72,6 +71,13 @@ Preconditions: --check chrome_processes_before_close ge 1 --check chrome_processes_after_close eq 0 ``` The page sets the cookie `session=abc123` and `localStorage.theme=dark` (`storage_has_page_values`); the restore adds the cookie `restored` and the key `restored_key`. +- **Closing a tab that is not focused (`S15.tabs`).** A `BrowserToolExecutor` opens `page2.html` in a new tab, switches back to the first tab, closes the new one and lists the tabs at once, five times. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --field run) + control-sdk program S15/close_tab --timeout 300 \ + --check setup_ok eq true --check all_closed eq true --check stale_lists eq 0 + ``` + Expected: once `browser_close_tab` has answered, `browser_list_tabs` lists only the remaining tab. Known failure (reproduced 2026-11-12, [#6785](https://github.com/OpenHands/software-agent-sdk/issues/6785)): the close answers when Chrome acknowledges `Target.closeTarget`, but browser-use's tab list drops the tab only when `Target.detachedFromTarget` is handled 10–30 ms later, so the immediate list still shows it (`stale_lists` 5 of 5; closing the focused tab usually hides the lag). Fix: [#6786](https://github.com/OpenHands/software-agent-sdk/pull/6786). - **No Chromium (`S15.degrade-on-failure`, `S15.chromium-detection`, `S15.prompt-guidance`).** The run's `HOME` has no Playwright cache, `PLAYWRIGHT_BROWSERS_PATH` is unset, and no Chromium is on `PATH` or in the system paths (on a machine with a system Chrome the program exits 3, since it cannot hide that one). A scripted agent with the terminal and the browser tool set still runs its terminal command. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -100,7 +106,7 @@ Preconditions: --check steps.blocked_domain.text not-contains "Navigated to: https://example.com/" \ --check steps.blocked_domain.is_error eq true --check steps.blocked_domain.text contains "blocked by security policy" ``` - Expected: the page outside `allowed_domains` is not opened, and the observation is an error that says the navigation was blocked by the security policy (#6641). + Expected: the page outside `allowed_domains` is not opened, and the observation is an error that says the navigation was blocked by the security policy. - **A hung page: timeouts, reset and recovery (`S15.session-recovery`, `S15.action-routing`, `S15.cleanup`).** With `action_timeout_seconds` 12, the agent opens a page whose script never yields, then makes three more calls. The third call triggers the session reset. Then it opens a normal page and reads its state. The program counts this run's Chromium processes after `conversation.close()`. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S16-planning-workflow-tools.md b/.agents/skills/verify-sdk/references/feature-map/S16-planning-workflow-tools.md index 88d75262c41..c3af9f786aa 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S16-planning-workflow-tools.md +++ b/.agents/skills/verify-sdk/references/feature-map/S16-planning-workflow-tools.md @@ -1,8 +1,8 @@ -# Planning, task tracking, oracle, ToM and workflow tools +# Planning, task tracking, oracle, ToM, workflow and reaction tools -Tools an SDK program adds to an agent to help it plan and coordinate rather than to touch the system directly. `task_tracker` keeps a structured task list (`plan` replaces it, `view` shows it, each item `todo`, `in_progress` or `done`) and saves it as `TASKS.json` in the conversation's persistence directory. The planning preset (`get_planning_agent(llm)`) is a read-only agent with `glob`, `grep` and `planning_file_editor`, a planning system prompt that fixes a five-section plan structure, and a larger summarizing condenser; `planning_file_editor` can view any file but edit only `PLAN.md`, which it creates with section headers at `.agents_tmp/PLAN.md` (or a legacy root `PLAN.md`, or an absolute `plan_path`). Example 24 chains a planning conversation and an execution conversation over the same workspace. `ask_oracle` sends the agent's question to a second model, the saved LLM profile named `oracle`, and returns its answer. `tom_consult` and `sleeptime_compute` wrap the `tom-swe` Theory-of-Mind agent: one gives personalized guidance for a vague request, the other indexes past conversations into a user model stored under the user persistence directory. `workflow` (from `workflow_tool_set`) runs a Python script the agent writes, `async def main(wf)`, in a restricted namespace where `wf.run_agent`, `wf.map_agents`, `wf.pipeline` and `wf.reduce_agent` fan work out to registered sub-agents with a concurrency cap and a one-hour limit. +Tools an SDK program adds to an agent to help it plan and coordinate rather than to touch the system directly. `task_tracker` keeps a structured task list (`plan` replaces it, `view` shows it, each item `todo`, `in_progress` or `done`) and saves it as `TASKS.json` in the conversation's persistence directory. The planning preset (`get_planning_agent(llm)`) is a read-only agent with `glob`, `grep` and `planning_file_editor`, a planning system prompt that fixes a five-section plan structure, and a larger summarizing condenser; `planning_file_editor` can view any file but edit only `PLAN.md`, which it creates with section headers at `.agents_tmp/PLAN.md` (or a legacy root `PLAN.md`, or an absolute `plan_path`). Example 24 chains a planning conversation and an execution conversation over the same workspace. `ask_oracle` sends the agent's question to a second model, the saved LLM profile named `oracle`, and returns its answer. `tom_consult` and `sleeptime_compute` wrap the `tom-swe` Theory-of-Mind agent: one gives personalized guidance for a vague request, the other indexes past conversations into a user model stored under the user persistence directory. `workflow` (from `workflow_tool_set`) runs a Python script the agent writes, `async def main(wf)`, in a restricted namespace where `wf.run_agent`, `wf.map_agents`, `wf.pipeline` and `wf.reduce_agent` fan work out to registered sub-agents with a concurrency cap and a one-hour limit. `react` and `unreact` (`openhands.tools.reaction`) let the agent put a small emoji (`eyes`, `+1`, `white_check_mark`, `tada`, `thinking_face`, `heart`) on a message, by default the user's latest one, as a sign it saw it; each successful call records a non-LLM `ReactionEvent` right after the tool's observation, and the program's user reacts with `conversation.react(emoji, target_event_id=None, removed=False)`, which by default targets the agent's latest message or `finish` and does not start a run. -Source: `openhands-tools/openhands/tools/task_tracker/`, `openhands-tools/openhands/tools/planning_file_editor/`, `openhands-tools/openhands/tools/preset/planning.py`, `openhands-tools/openhands/tools/ask_oracle/`, `openhands-tools/openhands/tools/tom_consult/`, `openhands-tools/openhands/tools/workflow/`, `openhands-sdk/openhands/sdk/context/prompts/sections/planning.py` +Source: `openhands-tools/openhands/tools/task_tracker/`, `openhands-tools/openhands/tools/planning_file_editor/`, `openhands-tools/openhands/tools/preset/planning.py`, `openhands-tools/openhands/tools/ask_oracle/`, `openhands-tools/openhands/tools/tom_consult/`, `openhands-tools/openhands/tools/workflow/`, `openhands-sdk/openhands/sdk/context/prompts/sections/planning.py`, `openhands-tools/openhands/tools/reaction/`, `openhands-sdk/openhands/sdk/event/reaction.py` Needs: `llm` (examples, real oracle and ToM calls, workflow sub-agents) @@ -30,6 +30,7 @@ Examples: `examples/01_standalone_sdk/24_planning_agent_workflow.py`, `examples/ - `S16.workflow-pipeline`: `wf.pipeline(items, stage1, stage2, ...)` runs sync or async stages per item with no barrier between stages, keeps item order, and drops a failed item to `None`; it needs at least one stage. Unit: `tests/tools/workflow/test_workflow_tool.py` (`test_pipeline_*`). Example: none. Real-LLM: none. - `S16.workflow-errors-and-output`: the script can catch common exceptions and `ExceptionGroup`s; uncaught errors come back formatted (with group details); the input `wf.reduce_agent` sends its sub-agent is cut to 12,000 characters, by whole trailing elements ("items omitted to fit") or characters; the script's own return value reaches the LLM whole. Unit: `tests/tools/workflow/test_workflow_tool.py` (`test_workflow_script_can_catch_*`, `test_format_exception_includes_exception_group_details`, `test_format_value_*`). Example: none. Real-LLM: none. - `S16.workflow-lifecycle`: closing the context closes the task manager once, and `run_agent` after close raises. Unit: `tests/tools/workflow/test_workflow_tool.py` (`test_workflow_context_close_propagates_to_manager`, `test_workflow_context_close_is_idempotent`, `test_run_agent_raises_after_close`). Example: none. Real-LLM: none. +- `S16.reactions`: `Tool(name=ReactTool.name)` / `Tool(name=UnreactTool.name)` (`react`, `unreact`) put an emoji on, or take it off, the user's latest message or the `message_id` given; each call returns `Reacted ::.` / `Removed ::.` and the agent records one agent `ReactionEvent` right after the observation; an emoji outside the six is rejected as a tool validation error. `conversation.react(emoji)` records a user `ReactionEvent` on the agent's latest answer (`finish` or message) with its text ending, raises `ValueError` for an unknown emoji, leaves the run status alone, and the agent sees it as one `[The user reacted ...]` line on its next turn, while its own reactions stay out of what it sees. Unit: `tests/tools/reaction/test_reaction_tools.py`. Example: none. Real-LLM: none. ## How to get to it (agent POV) @@ -38,6 +39,7 @@ Examples: `examples/01_standalone_sdk/24_planning_agent_workflow.py`, `examples/ - Oracle: save a profile first, `LLMProfileStore(dir).save("oracle", LLM(...), include_secrets=True)` (`ORACLE_PROFILE_NAME` from `openhands.tools.ask_oracle`), then `Agent(tools=[Tool(name=AskOracleTool.name)])` and `LocalConversation(..., profile_store_dir=dir)`. `S16.oracle-*`. - ToM: `from openhands.tools.tom_consult import TomConsultTool, SleeptimeComputeTool, SleeptimeComputeAction`; `Tool(name=TomConsultTool.name, params=tom_params)`; `OH_PERSISTENCE_DIR` moves the user-model store. `S16.tom-*`. - Workflow: `from openhands.tools.workflow import WorkflowToolSet`; `Tool(name=WorkflowToolSet.name)`; sub-agents come from `register_agent_if_absent(name=..., factory_func=..., description=...)` (`openhands.sdk.subagent`) or the built-ins; `WorkflowTool.create(executor=...)` injects a custom executor. `S16.workflow-*`. +- Reactions: `from openhands.tools.reaction import ReactTool, UnreactTool`; `Agent(tools=[Tool(name=ReactTool.name), Tool(name=UnreactTool.name)])` (not in any preset); `conversation.react("heart")`, `ReactionEvent` from `openhands.sdk.event`. `S16.reactions`. - Integration runner: `tests/integration/run_infer.py --tool-preset planning` swaps in `get_planning_tools()`. ## Driving it @@ -176,6 +178,32 @@ Preconditions: ``` The parent agent writes and runs a workflow that fans `coverage_auditor` sub-agents over the repository and prints the reducer's report. +- **Emoji reactions, scripted (`S16.reactions`).** A persisted conversation whose agent has only `react` and `unreact`. Turn 1 ("Please look into the build."): the agent reacts `eyes`, tries `rocket`, takes `eyes` back, reacts `white_check_mark`, and finishes. The program then calls `conversation.react("heart")` and `conversation.react("rocket")`; turn 2 ("Thanks, that's all.") finishes, with a recording LLM keeping what each call was sent. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --field run) + control-sdk program S16/reactions \ + --check status_turn1 eq finished --check status_after_user_react eq finished --check status_turn2 eq finished \ + --check reactions len-eq 4 --check event_before_first_reaction eq ObservationEvent \ + --check reactions.0.source eq agent --check reactions.0.emoji eq eyes --check reactions.0.on_first_message eq true \ + --check reactions.1.emoji eq eyes --check reactions.1.removed eq true --check reactions.1.on_first_message eq true \ + --check reactions.2.emoji eq white_check_mark --check reactions.2.on_first_message eq true \ + --check reactions.3.source eq user --check reactions.3.emoji eq heart --check reactions.3.on_first_finish eq true \ + --check reactions.3.text_ending eq "The build is green." \ + --check observations.0.text eq "Reacted :eyes:." --check observations.1.text eq "Removed :eyes:." \ + --check agent_errors len-eq 1 --check agent_errors.0 contains "Error validating tool 'react'" \ + --check bad_user_emoji contains "Unknown emoji 'rocket'" --check user_reaction_target_is_finish eq true \ + --check agent_reaction_preceded_by len-eq 3 --check agent_reaction_preceded_by.0 eq ObservationEvent \ + --check agent_reaction_preceded_by.1 eq ObservationEvent --check agent_reaction_preceded_by.2 eq ObservationEvent \ + --check turn2_sees_user_heart eq true --check turn2_user_reaction_lines eq 1 --check turn2_sees_own_reaction_line eq false + ``` +- **Emoji reactions with a real model (`S16.reactions`).** Live; the model gets `react` and `unreact` and one message asking it to react `eyes` first and then answer. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --llm deepseek-flash --field run) + control-sdk program S16/reactions --arg live \ + --check execution_status eq finished --check react_calls ge 1 \ + --check agent_reactions.0.emoji eq eyes --check agent_reactions.0.on_message eq true + ``` + ## Gotchas - `tests/integration/run_infer.py --tool-preset planning` (also offered by the integration-runner workflow) pairs the planning tools with task tests that edit files. The planner can write only its plan, so a task test that edits files, such as t01, fails by construction (on 2026-11-03, t01 with DeepSeek: "Typos were not fully corrected"). The map therefore drives the preset with the scripted program above, not through the runner. diff --git a/.agents/skills/verify-sdk/references/feature-map/S17-custom-and-client-tools.md b/.agents/skills/verify-sdk/references/feature-map/S17-custom-and-client-tools.md index 1808cbe3458..8920fb0dbca 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S17-custom-and-client-tools.md +++ b/.agents/skills/verify-sdk/references/feature-map/S17-custom-and-client-tools.md @@ -17,8 +17,9 @@ Examples: `examples/01_standalone_sdk/02_custom_tools.py`, `examples/01_standalo - `S17.auto-naming`: `ToolDefinition.name` is derived from the class name (`FileEditorTool` to `file_editor`) unless the class sets `name`. Unit: `tests/cross/test_automatic_naming.py`. Example: `examples/01_standalone_sdk/02_custom_tools.py` (`BashAndGrepToolSet.name`). Real-LLM: none. - `S17.auto-registration`: importing a built-in tool module (or `openhands.tools`) registers its tools, after which specs resolve. Unit: `tests/cross/test_automatic_registration.py`, `tests/tools/test_init.py`. Example: none. Real-LLM: none. - `S17.tool-spec`: `Tool(name, params)` copies its `params` (later changes to the caller's dict do not reach it), round-trips through JSON, and rejects an empty name; it is not frozen (attributes can be reassigned). Unit: `tests/sdk/tool/test_tool.py`. Example: `examples/01_standalone_sdk/02_custom_tools.py`. Real-LLM: none. -- `S17.agent-tool-init`: the agent turns its specs into tools at conversation start, adds the built-in `finish` (and `think` only when `include_default_tools` opts into it; it is no longer a default since #6159), rejects duplicate tool names, and does not let a runtime tool replace an existing one; a custom tool may replace `finish`. Unit: `tests/sdk/agent/test_agent_tool_init.py`. Example: `examples/01_standalone_sdk/02_custom_tools.py`. Real-LLM: none. +- `S17.agent-tool-init`: the agent turns its specs into tools at conversation start, adds the built-in `finish` (and `think` only when `include_default_tools` opts into it), rejects duplicate tool names, and does not let a runtime tool replace an existing one; a custom tool may replace `finish`. Unit: `tests/sdk/agent/test_agent_tool_init.py`. Example: `examples/01_standalone_sdk/02_custom_tools.py`. Real-LLM: none. - `S17.llm-schemas`: `to_openai_tool()`, `to_responses_tool()` (with `strict` and parameters) and `to_mcp_tool()` render the action schema without the internal `kind` field, keep required fields and annotations, and expand nested and circular `$ref`s without recursion errors. Unit: `tests/sdk/tool/test_mcp_schema.py`, `tests/sdk/tool/test_to_responses_tool.py`, `tests/sdk/tool/test_tool_definition.py` (`test_to_mcp_tool_*`, `test_mcp_tool_schema_required_fields`). Example: none. Real-LLM: none. +- `S17.schema-constraints`: the tool schema sent to the model keeps the JSON Schema constraints the `Action` model declares: a field's `pattern`, a one-value `Literal` (`const`), and `additionalProperties: false` from `extra="forbid"`. Unit: **none (untested)**. Example: none. Real-LLM: none. - `S17.injected-fields`: a `security_risk` parameter is added to tools that are not read-only (optional, ahead of content parameters, gated by the security setting) and a `summary` parameter is always added, while an MCP tool's own description is kept. Unit: `tests/sdk/tool/test_tool_definition.py` (`test_security_risk_only_added_for_non_readonly_tools`, `test_security_risk_is_optional_field_in_schema`, `test_security_risk_precedes_content_params_in_schema`), `tests/sdk/tool/test_to_responses_tool_security.py`, `tests/sdk/tool/test_to_responses_tool_summary.py`. Example: none. Real-LLM: none. - `S17.schema-serialization`: `Action`/`Observation` subclasses are frozen and serialize as a discriminated union by `kind`, including nested polymorphic fields; tool definitions are immutable and `set_executor` returns a copy. Unit: `tests/sdk/tool/test_tool_serialization.py`, `tests/sdk/tool/test_schema_immutability.py`, `tests/sdk/tool/test_tool_immutability.py`. Example: none. Real-LLM: none. - `S17.py-type`: `py_type()` maps JSON-schema types (string, integer, number, boolean, object, arrays, nullable unions) to Python types for dynamically built actions. Unit: `tests/sdk/tool/test_py_type.py`. Example: none. Real-LLM: none. @@ -154,7 +155,19 @@ Preconditions: --check no_executor_call contains "has no executor" --check no_executor_as_executable contains "has no executor" \ --check wrong_observation_type contains "instance of WordCountObservation" ``` - `security_risk` goes only to the tool that is not read-only, ahead of `summary` and the tool's own fields. The `Section` outline refers to itself; it is sent expanded one level, with no `$ref`. An executor that returns another observation class fails with a `ValidationError` that names the expected class (`Input should be a valid dictionary or instance of WordCountObservation`, `wrong_observation_type`; the bare `AttributeError ... 'pop'` was fixed by #5798). + `security_risk` goes only to the tool that is not read-only, ahead of `summary` and the tool's own fields. The `Section` outline refers to itself; it is sent expanded one level, with no `$ref`. An executor that returns another observation class fails with a `ValidationError` that names the expected class (`Input should be a valid dictionary or instance of WordCountObservation`, `wrong_observation_type`). +- **Constraints the `Action` declares (`S17.schema-constraints`).** `IssueRefundAction` has `order_id: str = Field(pattern=r"^ORD-[0-9]{6}$")`, `currency: Literal["USD"]` and `model_config = ConfigDict(extra="forbid")`; the program renders its tool with `to_openai_tool()` and `to_responses_tool()` and compares them with pydantic's own schema. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --field run) + control-sdk program S17/schema_constraints \ + --check pydantic.order_id_pattern eq '^ORD-[0-9]{6}$' --check pydantic.currency_const eq USD \ + --check pydantic.additional_properties eq false \ + --check openai.order_id_pattern eq '^ORD-[0-9]{6}$' --check openai.currency_const eq USD \ + --check openai.additional_properties eq false \ + --check responses.order_id_pattern eq '^ORD-[0-9]{6}$' --check responses.currency_const eq USD \ + --check responses.additional_properties eq false + ``` + Expected: the parameters the model gets keep `pattern`, the `const` (or a one-value `enum`) and `additionalProperties: false`. Known failure (reproduced 2026-11-12, [#6766](https://github.com/OpenHands/software-agent-sdk/issues/6766)): pydantic's schema has all three, but both rendered tools send `order_id` as `{"type": "string", "description": ...}`, `currency` as a plain string and no `additionalProperties`: `_process_schema_node` in `openhands/sdk/tool/schema.py` keeps only `type`, `description`, `properties`, `required`, `items`, `enum` and the first `anyOf` branch. - **Validating a dumped action twice (`S17.schema-serialization`).** The program validates `action.model_dump(mode="json")` back into an `Action` and then validates the same dict again. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -162,7 +175,7 @@ Preconditions: --check action_round_trip_type eq AddNoteAction \ --check validate_keeps_input_kind eq true --check validate_same_dict_again eq "no error" ``` - Expected: validation reads the caller's dict without changing it. Fixed: a known failure on 2026-10-13 (the validator popped `kind` from the caller's dict, so a second validation failed); it passes since #5770, re-verified 2026-11-03. + Expected: validation reads the caller's dict without changing it. - **Built-in tools and duplicates at agent start (`S17.agent-tool-init`).** Agents with `word_count` and default, narrowed, empty and unknown `include_default_tools`; a duplicate spec; a custom tool named `finish` with only `ThinkTool` built in; and a runtime tool that would replace it. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -236,5 +249,5 @@ Preconditions: - A client tool whose name matches a built-in the agent attaches (for example `finish`) is rejected when the conversation starts the agent, by the duplicate-name check (`ValueError: Duplicate tool names found: {'finish'}`), not by `ClientToolRegistrationError`; that error is for names already in the tool registry (such as `terminal` once imported) and for duplicates within one `client_tools` list. - A client tool argument named `kind` is stored as `mcp_arg_kind`, and `event.action.model_dump()` adds `kind: "ClientAction_"`. A callback that forwards `model_dump()` (as example 53's does) sends the client renamed and extra keys; forward `json.loads(event.tool_call.arguments)` to pass on exactly what the model sent. - Client tool arguments are not checked against the spec's `enum`: a value outside it is dispatched like any other, so the client must validate. -- An executor that returns the wrong observation class fails with a `ValidationError` that names the expected type (since #5798; before, a bare `AttributeError ... 'pop'`). +- An executor that returns the wrong observation class fails with a `ValidationError` that names the expected type. - `Action.model_validate(d)` (and every discriminated model's validation) removes `kind` from the dict `d` it is given; copy the dict first if you need it again. diff --git a/.agents/skills/verify-sdk/references/feature-map/S18-workspace-backends.md b/.agents/skills/verify-sdk/references/feature-map/S18-workspace-backends.md index b9505177dd8..8e5a5b77e3a 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S18-workspace-backends.md +++ b/.agents/skills/verify-sdk/references/feature-map/S18-workspace-backends.md @@ -96,7 +96,7 @@ Preconditions: export SDK_VERIFY_RUN=$(control-sdk new --field run) control-sdk program S18/remote_workspace --arg ops --check path_working_dir_error missing --check type eq RemoteWorkspace ``` - Known failure (reproduced 2026-10-13, not yet filed): `Workspace(host=..., working_dir=Path(...))` (and `RemoteWorkspace(working_dir=Path(...))`) raises `pydantic_core.ValidationError: 1 validation error for RemoteWorkspace working_dir Input should be a valid string [type=string_type, input_type=PosixPath]`. `RemoteWorkspaceMixin` redeclares `working_dir: str` without the `BeforeValidator` that `BaseWorkspace` uses to accept a `Path`; the local branch accepts one. + Known failure (reproduced 2026-10-13, [#6654](https://github.com/OpenHands/software-agent-sdk/issues/6654)): `Workspace(host=..., working_dir=Path(...))` (and `RemoteWorkspace(working_dir=Path(...))`) raises `pydantic_core.ValidationError: 1 validation error for RemoteWorkspace working_dir Input should be a valid string [type=string_type, input_type=PosixPath]`. `RemoteWorkspaceMixin` redeclares `working_dir: str` without the `BeforeValidator` that `BaseWorkspace` uses to accept a `Path`; the local branch accepts one. - **Settings from the server (`S18.remote-settings`).** LLM settings, an LLM profile, two secrets and an MCP server are stored through the server's settings API; the workspace reads them back. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S21-acp-agent.md b/.agents/skills/verify-sdk/references/feature-map/S21-acp-agent.md index 998195da594..2c429b03fcd 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S21-acp-agent.md +++ b/.agents/skills/verify-sdk/references/feature-map/S21-acp-agent.md @@ -1,6 +1,6 @@ # ACP agent (in-process) -A Python program can swap the built-in LLM loop for an external coding agent that speaks the Agent Client Protocol: `ACPAgent(acp_command=[...])` (for example `ACP_PROVIDERS["claude-code"].default_command`, an `npx` launch of `claude-agent-acp`) is passed to an ordinary `Conversation`. On the first `send_message()`/`run()` the SDK spawns that subprocess in the workspace, runs the `initialize` / `authenticate` / `new_session` (or `load_session` to resume) handshake, sets the session mode and model, and forwards the agent's `mcp_config`. Each `run()` sends one ACP `prompt` built from the user message (text and images, plus a one-time context suffix with skills, repo context and secret names) and turns the server's streamed `session_update` notifications into SDK events: tool-call events as they happen, then a `FinishAction` carrying the reply and reasoning, with token usage and cost recorded on a sentinel `LLM`'s metrics. OpenHands tools, condensers, critics and confirmation do not apply; the ACP server owns its tools, context window and permissions (the SDK auto-approves every permission request). `conversation.ask_agent(...)` forks the ACP session for a side question, `set_acp_model` / `Conversation.switch_acp_model` switch the model at runtime, secrets reach the subprocess env and are masked in everything relayed back, and `agent.close()` terminates the subprocess. Providers known to the registry: `claude-code`, `codex`, `gemini-cli`, `kimi-code`, `pi`, `opencode`, or a custom command. +A Python program can swap the built-in LLM loop for an external coding agent that speaks the Agent Client Protocol: `ACPAgent(acp_command=[...])` (for example `ACP_PROVIDERS["claude-code"].default_command`, an `npx` launch of `claude-agent-acp`) is passed to an ordinary `Conversation`. On the first `send_message()`/`run()` the SDK spawns that subprocess in the workspace, runs the `initialize` / `authenticate` / `new_session` (or `load_session` to resume) handshake, sets the session mode, model and reasoning effort, and forwards the agent's `mcp_config`. Each `run()` sends one ACP `prompt` built from the user message (text and images, plus a one-time context suffix with skills, repo context and secret names) and turns the server's streamed `session_update` notifications into SDK events: tool-call events as they happen, then a `FinishAction` carrying the reply and reasoning, with token usage and cost recorded on a sentinel `LLM`'s metrics. OpenHands tools, condensers, critics and confirmation do not apply; the ACP server owns its tools, context window and permissions (the SDK auto-approves every permission request). `conversation.ask_agent(...)` forks the ACP session for a side question, `set_acp_model` / `Conversation.switch_acp_model` switch the model and `set_acp_effort` / `Conversation.switch_acp_effort` the reasoning effort at runtime, secrets reach the subprocess env and are masked in everything relayed back, and `agent.close()` terminates the subprocess. Providers known to the registry: `claude-code`, `codex`, `gemini-cli`, `kimi-code`, `pi`, `opencode`, or a custom command. Source: `openhands-sdk/openhands/sdk/agent/acp_agent.py`, `openhands-sdk/openhands/sdk/agent/acp_contracts.py`, `openhands-sdk/openhands/sdk/agent/acp_models.py`, `openhands-sdk/openhands/sdk/agent/acp_model_discovery.py`, `openhands-sdk/openhands/sdk/agent/acp_file_credentials.py`, `openhands-sdk/openhands/sdk/agent/acp_tracing.py`, `openhands-sdk/openhands/sdk/settings/acp_providers.py`, `openhands-sdk/openhands/sdk/settings/acp_install_catalog.py`, `openhands-sdk/openhands/sdk/settings/model.py` @@ -12,7 +12,7 @@ Examples: `examples/01_standalone_sdk/40_acp_agent_example.py` - `S21.construct`: `ACPAgent` requires `acp_command`, defaults to a sentinel LLM, no tools and no default tools, is frozen, stores `acp_args`, and propagates `acp_model` to the sentinel LLM's model and metrics; it accepts `mcp_config` and an `agent_context`, while custom tools or a condenser raise `NotImplementedError` in `init_state`. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPAgentInstantiation`, `::TestACPAgentValidation`, `::TestACPAgentSerialization`. Example: `examples/01_standalone_sdk/40_acp_agent_example.py`. Real-LLM: none. - `S21.provider-registry`: `ACP_PROVIDERS` describes each provider (default `npx` command, session-mode, model config mechanism, file secrets, data-dir env var); providers are detected by agent name or command; the install catalog pins package versions that match the registry and renders Docker install plans. Unit: `tests/sdk/settings/test_acp_providers.py`, `tests/sdk/settings/test_acp_install_catalog.py`; live: `tests/sdk/settings/test_acp_install_catalog_pins_live.py` (`-m acp_live`). Example: `examples/01_standalone_sdk/40_acp_agent_example.py`. Real-LLM: none. -- `S21.settings`: `ACPAgentSettings(acp_server=..., acp_model=..., acp_command=...)` resolves the default or pinned-binary command (custom servers need an explicit command), carries the provider key and `acp_isolate_data_dir`, and `create_agent()` returns an `ACPAgent`. Unit: `tests/sdk/test_settings.py` (`test_acp_*`). Example: none. Real-LLM: none. +- `S21.settings`: `ACPAgentSettings(acp_server=..., acp_model=..., acp_effort=..., acp_command=...)` resolves the default or pinned-binary command (custom servers need an explicit command), carries the provider key, `acp_effort` and `acp_isolate_data_dir`, and `create_agent()` returns an `ACPAgent` with them. Unit: `tests/sdk/test_settings.py` (`test_acp_*`). Example: none. Real-LLM: none. - `S21.handshake-live`: every built-in provider's real `npx` server completes initialize, authenticate (with a bogus key), new_session, session mode and model selection, and reports models consistent with the registry. Unit: `tests/sdk/agent/test_acp_conformance.py` (`-m acp_live`, needs `npx` and the npm registry). Example: none. Real-LLM: none. - `S21.init-state`: `init_state` starts the server, emits a placeholder `SystemPromptEvent` with the context suffix as dynamic context (including secret names), and surfaces auth-required, spawn and other init failures as a classified `ConversationErrorEvent` with truncated detail before re-raising; a failed init leaves the agent reusable. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPAgentInitState`, `::TestClassifyACPInitError`, `::TestACPErrorDetail`, `::TestACPAgentCleanup::test_failed_init_keeps_agent_reusable`. Example: none. Real-LLM: none. - `S21.auth-selection`: `_select_auth_method` prefers file-backed logins (`chat-gpt` with a ChatGPT `auth.json`, `vertex-ai` with `GOOGLE_APPLICATION_CREDENTIALS`, `oauth-personal`) over API keys present in the env, and logs why no method could be performed. Unit: `tests/sdk/agent/test_acp_agent.py::TestSelectAuthMethod`, `::TestPreconfiguredCredentials`, `::TestAuthSelectionFailureReason`, `::TestUnperformableAuthMethodLogging`. Example: none. Real-LLM: none. @@ -22,9 +22,11 @@ Examples: `examples/01_standalone_sdk/40_acp_agent_example.py` - `S21.timeouts-retry`: a prompt is aborted only after `acp_prompt_timeout` seconds without any server activity (activity resets the clock); startup has a hard `acp_startup_timeout`; connection errors and retriable ACP server errors are retried a bounded number of times, timeouts and other errors are not. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPPromptIdleTimeout`, `::TestACPActivityHeartbeat`, `::TestACPPromptRetry`, `::TestClassifyACPTurnError`. Example: none. Real-LLM: none. - `S21.cancel-pause`: cancelling `arun()` mid-prompt sends `session/cancel`, drains the prompt, emits failed tool calls, pauses without a turn error, and restarts the session on the next turn if the drain times out. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPAgentAstep` (`test_astep_*cancel*`, `test_cleanup_interruption_*`). Example: none. Real-LLM: none. - `S21.session-resume`: the ACP session id is persisted in `state.agent_state` and used with `load_session` when the conversation is reloaded; `acp_resume_session_id` overrides it; a failed load falls back to a new session; session ids are redacted in logs and serialization. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPSessionIdPersistence`, `::TestHasLiveACPSession`. Example: none. Real-LLM: none. -- `S21.model-selection`: `acp_model` is applied through `set_config_option` (codex, claude; codex splits reasoning effort) or `set_session_model`, reapplied on resume, without silent fallback; `current_model_id` and `available_models` come from the server's session response. Unit: `tests/sdk/agent/test_acp_agent.py::TestMaybeSetSessionModel`, `::TestReapplySessionModelOnResume`, `::TestConfigOptionModelMechanism`, `::TestApplyAcpModel`, `::TestApplyAcpModelNoFallback`, `::TestExtractSessionModels`, `::TestACPAgentCurrentModelIdProperty`, `::TestACPAgentAvailableModelsProperty`, `tests/sdk/agent/test_acp_contracts.py`. Example: none. Real-LLM: none. +- `S21.model-selection`: `acp_model` is applied through `set_config_option` (codex, claude) or `set_session_model`, reapplied on resume, without silent fallback; `current_model_id` and `available_models` come from the server's session response, and `current_model_id` is the bare model id (a legacy `/` id for a provider with an effort option is split into the model and the effort). Unit: `tests/sdk/agent/test_acp_agent.py::TestMaybeSetSessionModel`, `::TestReapplySessionModelOnResume`, `::TestConfigOptionModelMechanism`, `::TestApplyAcpModel`, `::TestApplyAcpModelNoFallback`, `::TestExtractSessionModels`, `::TestACPAgentCurrentModelIdProperty`, `::TestACPAgentAvailableModelsProperty`, `tests/sdk/agent/test_acp_contracts.py`. Example: none. Real-LLM: none. - `S21.model-switch`: `agent.set_acp_model(model)` switches a live session (an invalid model or unsupported provider raises `ValueError`); `Conversation.switch_acp_model(...)` defers before the session starts, persists the authoritative model and refreshes `current_model_id`. Unit: `tests/sdk/agent/test_acp_agent.py::TestSetACPModel`, `::TestACPAgentSupportsRuntimeModelSwitch`, `tests/sdk/conversation/test_switch_model.py` (`-k acp`). Example: none. Real-LLM: none. -- `S21.model-discovery`: `discover_acp_models(settings, secrets=...)` starts the server in a throwaway session and reports its default and offered models (config options or the legacy models block), passing secrets and reporting auth and spawn failures. Unit: `tests/sdk/agent/test_acp_model_discovery.py` (runs `tests/sdk/agent/fake_acp_models_server.py` as a real subprocess). Example: none. Real-LLM: none. +- `S21.model-discovery`: `discover_acp_models(settings, secrets=...)` starts the server in a throwaway session and reports its default and offered models (config options or the legacy models block), the default model's `current_effort` and `available_efforts`, and `efforts_by_model` (the levels each model offers), passing secrets and reporting auth and spawn failures. Unit: `tests/sdk/agent/test_acp_model_discovery.py` (runs `tests/sdk/agent/fake_acp_models_server.py` as a real subprocess). Example: none. Real-LLM: none. +- `S21.effort`: `ACPAgent(acp_effort=...)` sets the reasoning effort after the model through the session config option the server tags `thought_level`, and `agent.current_effort` / `available_efforts` report what the session runs and offers for its current model; a level the model does not offer is skipped with a warning (the turn still runs at the server's default), and an explicit `acp_effort` wins over a legacy `/` suffix. Unit: `tests/sdk/agent/test_acp_model_discovery.py::test_session_start_applies_the_requested_effort`, `::test_effort_is_skipped_for_a_model_that_offers_none`, `tests/sdk/agent/test_acp_contracts.py::TestSessionEffortContract`, `tests/sdk/agent/test_acp_agent.py::TestSplitEffortSuffix`. Example: none. Real-LLM: none. +- `S21.effort-switch`: `Conversation.switch_acp_effort(effort)` before the first run only persists `acp_effort` (no subprocess starts) and the first session applies it; on a live session it switches the effort in place (same session) and persists `acp_effort` and the `acp_current_effort` hint in `base_state.json`; a level the model does not offer, a blank level, a model that offers no effort and a non-ACP conversation raise `ValueError` and change nothing; after `switch_acp_model` the effort follows the new model's options, and switching back re-applies the pinned effort. Unit: `tests/sdk/conversation/test_switch_model.py::test_switch_acp_effort_before_session_persists`, `::test_switch_acp_effort_live_applies_and_persists`, `::test_switch_acp_effort_rejected_keeps_persisted_state`, `::test_switch_acp_effort_rejects_non_acp_agent`, `tests/sdk/agent/test_acp_model_discovery.py::test_switches_effort_at_runtime`, `::test_model_switch_keeps_the_pinned_effort`. Example: none. Real-LLM: none. - `S21.ask-agent`: `conversation.ask_agent(question)` forks the ACP session, prompts the fork, records its token usage and cleans the fork state, leaving the main session untouched; it raises before the session exists. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPAgentAskAgent`, `::TestClientForkTextRouting`. Example: `examples/01_standalone_sdk/40_acp_agent_example.py`. Real-LLM: none. - `S21.usage-cost`: token usage and cost from `usage_update` notifications (or estimated from tokens) are recorded on the sentinel LLM's metrics per turn, incrementally, with latency and context window; partial metrics are kept on a usage timeout. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPAgentTelemetry`, `::TestExtractTokenUsage`, `::TestEstimateCostFromTokens`. Example: `examples/01_standalone_sdk/40_acp_agent_example.py` (prints `EXAMPLE_COST`). Real-LLM: none. - `S21.mcp-forwarding`: the agent's `mcp_config` is converted to ACP stdio or HTTP/SSE servers (with headers) and passed to `new_session` and `load_session`; none gives an empty list. Unit: `tests/sdk/agent/test_acp_agent.py::TestMcpConfigToAcpServers`, `::TestACPMcpForwarding`. Example: none. Real-LLM: none. @@ -37,10 +39,10 @@ Examples: `examples/01_standalone_sdk/40_acp_agent_example.py` ## How to get to it (agent POV) -- `from openhands.sdk.agent import ACPAgent`; `ACPAgent(acp_command=[...], acp_args=[...], acp_server="claude-code", acp_model="sonnet", acp_session_mode=None, acp_prompt_timeout=1800.0, acp_startup_timeout=90.0, acp_resume_session_id=None, acp_isolate_data_dir=False, acp_file_secrets=[...], mcp_config={...}, agent_context=AgentContext(...))`. Covers most sub-features. +- `from openhands.sdk.agent import ACPAgent`; `ACPAgent(acp_command=[...], acp_args=[...], acp_server="claude-code", acp_model="sonnet", acp_effort="high", acp_session_mode=None, acp_prompt_timeout=1800.0, acp_startup_timeout=90.0, acp_resume_session_id=None, acp_isolate_data_dir=False, acp_file_secrets=[...], mcp_config={...}, agent_context=AgentContext(...))`. Covers most sub-features. - `from openhands.sdk.settings.acp_providers import ACP_PROVIDERS`; `ACP_PROVIDERS[key].default_command` (`S21.provider-registry`). - `ACPAgentSettings(acp_server=..., ...).create_agent()` and `discover_acp_models(settings, secrets=...)` (`S21.settings`, `S21.model-discovery`). -- `Conversation(agent=acp_agent, workspace=cwd, secrets={...}, persistence_dir=...)`, `conversation.send_message(Message(... ImageContent(image_urls=[...])))`, `conversation.run()` / `await conversation.arun()`, `conversation.ask_agent(q)`, `conversation.switch_acp_model(...)`; `agent.set_acp_model(...)`, `agent.current_model_id`, `agent.available_models`, `agent.llm.metrics`, `agent.close()`. +- `Conversation(agent=acp_agent, workspace=cwd, secrets={...}, persistence_dir=...)`, `conversation.send_message(Message(... ImageContent(image_urls=[...])))`, `conversation.run()` / `await conversation.arun()`, `conversation.ask_agent(q)`, `conversation.switch_acp_model(...)`, `conversation.switch_acp_effort(...)`; `agent.set_acp_model(...)`, `agent.set_acp_effort(...)`, `agent.current_model_id`, `agent.available_models`, `agent.current_effort`, `agent.available_efforts`, `agent.llm.metrics`, `agent.close()`. - Env for the default Claude Code provider: `ANTHROPIC_API_KEY` and optionally `ANTHROPIC_BASE_URL` (Codex: `CODEX_API_KEY` / `OPENAI_API_KEY` or `~/.codex/auth.json`; Gemini: `GEMINI_API_KEY` or `GOOGLE_APPLICATION_CREDENTIALS`). - Tests marker: `pytest -m acp_live` selects the live conformance and pin tests. @@ -48,7 +50,7 @@ Examples: `examples/01_standalone_sdk/40_acp_agent_example.py` Preconditions: - `uv sync --dev` has run; `control-sdk` is on `PATH` (`export PATH="$PWD/.agents/skills/verify-sdk/scripts:$PATH"`). `sdk-map run` does both. -- The scripted bullets need nothing else. The ACP server is `programs/S21/fake_acp.py`, a real stdio JSON-RPC subprocess that the SDK spawns through `ACPAgent(acp_command=[python, fake_acp.py])`. It streams thoughts, tool calls with progress, a permission request, reply chunks and `usage_update`s; behaves according to a tag in the prompt (`[slow]`, `[hang]`, `[flaky]`, `[fail]`, `[cancel]`, `[secret]`, `[big]`, `[silent]`); and journals every request it receives, so a program can check what the SDK sent as well as what it made of the replies. +- The scripted bullets need nothing else. The ACP server is `programs/S21/fake_acp.py`, a real stdio JSON-RPC subprocess that the SDK spawns through `ACPAgent(acp_command=[python, fake_acp.py])`. It reports models `m1` and `m2` as a `model` config option (with `FAKE_ACP_EFFORT` set, `m1` also offers a `thought_level` effort option); it streams thoughts, tool calls with progress, a permission request, reply chunks and `usage_update`s; behaves according to a tag in the prompt (`[slow]`, `[hang]`, `[flaky]`, `[fail]`, `[cancel]`, `[secret]`, `[big]`, `[silent]`); and journals every request it receives, so a program can check what the SDK sent as well as what it made of the replies. - The live handshake bullet needs `npx` (Node 22 here) and the npm registry (exit 3 when either is missing). Example 40 needs `ANTHROPIC_API_KEY`. - **A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model="m2", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={"FAKE_TOKEN": ..., "EMPTY_SECRET": ""}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model("m1")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment. @@ -101,7 +103,7 @@ Preconditions: --check ask_prompt_tokens_delta eq 100 \ --check ask_cost_delta eq 0.05 --check ask_warnings len-eq 0 --check ask_seconds lt 1.5 ``` - Known failure (reproduced 2026-10-13, not yet filed): the fork's cost is lost and every `ask_agent` call stalls. The client bridge (`_OpenHandsACPBridge.session_update`) routes every update for the fork session to the fork branch, which keeps only message chunks and `return`s before the `UsageUpdate` branch. As a result: + Known failure (reproduced 2026-11-12, [#6646](https://github.com/OpenHands/software-agent-sdk/issues/6646)): the fork's cost is lost and every `ask_agent` call stalls. The client bridge (`_OpenHandsACPBridge.session_update`) routes every update for the fork session to the fork branch, which keeps only message chunks and `return`s before the `UsageUpdate` branch. As a result: - `ask_agent` waits the full `ACP_USAGE_UPDATE_TIMEOUT` and logs `UsageUpdate not received within 2.0s for fork session ...`. - It records no cost: `ask_cost_delta` is 0.0, `ask_seconds` is 2.01, and `ask_warnings` holds that line. - Token usage from the prompt response is still recorded. @@ -156,7 +158,7 @@ Preconditions: --check cases.hang.prompts_sent eq 1 --check cases.hang.cancel_sent eq 1 --check cases.hang.status eq error \ --check cases.hang.seconds lt 5 ``` - Known failure (reproduced 2026-10-13, not yet filed): a timed-out turn is sent again. `ACPAgent._step` handles the idle timeout by emitting the timeout message and setting `execution_status = ERROR` without raising. `LocalConversation.run()` stops only on `PAUSED`, `STUCK`, `FINISHED` or an exception, so the loop calls `step()` again, which re-sends the same user message. Here the server received 3 prompts (and 2–3 `session/cancel`s) over 6 s, until the stuck detector set `stuck`: `prompts_sent` 3, `status` `stuck`. With the default `acp_prompt_timeout` of 1800 s that is 90 minutes. With stuck detection off, the loop runs until `max_iteration_per_run`. + Known failure (reproduced 2026-11-12, [#6645](https://github.com/OpenHands/software-agent-sdk/issues/6645)): a timed-out turn is sent again. `ACPAgent._step` handles the idle timeout by emitting the timeout message and setting `execution_status = ERROR` without raising. `LocalConversation.run()` stops only on `PAUSED`, `STUCK`, `FINISHED` or an exception, so the loop calls `step()` again, which re-sends the same user message. Here the server received 3 prompts (and 2–3 `session/cancel`s) over 6 s, until the stuck detector set `stuck`: `prompts_sent` 3, `status` `stuck`. With the default `acp_prompt_timeout` of 1800 s that is 90 minutes. With stuck detection off, the loop runs until `max_iteration_per_run`. - **Cancel a running turn (`S21.cancel-pause`).** `arun()` runs as a task. The server starts a long tool call and waits, and the program cancels the task. Then it runs another turn. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -166,7 +168,7 @@ Preconditions: --check next_error missing --check next_status eq finished --check next_finishes eq 1 --check sessions_used eq 1 ``` The SDK sends `session/cancel` and drains the prompt (the server answers `cancelled`). It closes the in-flight tool card as failed and pauses without a turn error. The next turn reuses the session. -- **A Codex binary with file credentials (`S21.file-secrets`, `S21.auth-selection`, `S21.model-selection`, `S21.secrets-env`).** The server is launched by path as `/bin/codex-acp`, the form a pinned install takes. It reports `codex-acp` and offers the auth methods `chat-gpt` and `api-key`. `acp_model="gpt-5.5/high"`, and `CODEX_HOME` points at an ambient directory. There are two runs: +- **A Codex binary with file credentials (`S21.file-secrets`, `S21.auth-selection`, `S21.model-selection`, `S21.secrets-env`).** The server is launched by path as `/bin/codex-acp`, the form a pinned install takes. It reports `codex-acp` and offers the auth methods `chat-gpt` and `api-key`. `acp_model="gpt-5.5/high"` (the legacy `/` form), and `CODEX_HOME` points at an ambient directory. There are two runs: - `CODEX_API_KEY` only, no isolation. - A ChatGPT `CODEX_AUTH_JSON` plus `CODEX_API_KEY` and a Gemini `GOOGLE_APPLICATION_CREDENTIALS_JSON`, with `acp_isolate_data_dir=True`. ```sh @@ -175,7 +177,7 @@ Preconditions: --check runs.api_key.status eq finished --check runs.api_key.auth_methods_used.0 eq api-key \ --check runs.api_key.codex_home_is_ambient eq true --check runs.api_key.env_has_codex_api_key eq true \ --check runs.api_key.config_calls.0 eq model=gpt-5.5 --check runs.api_key.config_calls.1 eq reasoning_effort=high \ - --check runs.api_key.session_modes.0 eq agent-full-access --check runs.api_key.current_model_id eq gpt-5.5/high \ + --check runs.api_key.session_modes.0 eq agent-full-access --check runs.api_key.current_model_id eq gpt-5.5 \ --check runs.api_key.supports_switch eq true \ --check runs.chatgpt_file.status eq finished --check runs.chatgpt_file.auth_methods_used.0 eq chat-gpt \ --check runs.chatgpt_file.codex_home_is_ambient eq false --check runs.chatgpt_file.codex_home_under_conversation eq true \ @@ -184,7 +186,7 @@ Preconditions: --check runs.chatgpt_file.acp_dir.0 eq codex ``` For Codex: - - The combined model id is split into `model` and `reasoning_effort` config options, and the session mode is `agent-full-access`. + - The combined model id is split into `model` and `reasoning_effort` config options (the fake tags `reasoning_effort` with category `thought_level`, as codex-acp does), `current_model_id` is the bare `gpt-5.5`, and the session mode is `agent-full-access`. - A ChatGPT `auth.json` wins over the API key. - With isolation, `CODEX_HOME` moves to `/acp/codex`, where `auth.json` is written with mode 0600; the blob itself never reaches the env. - Gemini's credential is not materialised for Codex. @@ -207,6 +209,50 @@ Preconditions: --check created.type eq ACPAgent --check created.acp_model eq m2 --check created.isolate eq true \ --check created_status eq finished --check created_current_model eq m2 ``` +- **Reasoning effort (`S21.effort`, `S21.effort-switch`, `S21.model-discovery`, `S21.settings`).** `programs/S21/acp_effort.py` runs `fake_acp.py` with `FAKE_ACP_EFFORT=1`: model `m1` offers a `thought_level` select (`effort`: `low`/`medium`/`high`, server default `medium`, reset by a model switch) and `m2` offers none. It drives: + - `ACPAgent(acp_model="m1", acp_effort="high")` for one turn, then `switch_acp_effort("low")` live, `"ultra"` and a blank level, `switch_acp_model("m2")`, `switch_acp_effort("low")` on `m2`, `switch_acp_model("m1")`, and another turn; + - `acp_effort="ultra"` at construction, which `m1` does not offer; + - `switch_acp_effort("high")` on a conversation that has not run, then its first turn; + - a server named `codex-acp` with `acp_model="gpt-5.5/high"`, alone and with `acp_effort="low"`; + - `switch_acp_effort` on a regular `Agent` conversation; + - `discover_acp_models` and `ACPAgentSettings(acp_effort="low")`. + ```sh + export SDK_VERIFY_RUN=$(control-sdk new --field run) + control-sdk program S21/acp_effort --timeout 180 \ + --check start_error missing --check start.config_calls.0 eq model=m1 --check start.config_calls.1 eq effort=high \ + --check start.current_effort eq high --check start.available_efforts len-eq 3 --check start.hint_effort eq high \ + --check switched.config_calls.0 eq effort=low --check switched.current_effort eq low --check switched.acp_effort eq low \ + --check switched.same_session eq true --check switched.persisted.acp_effort eq low --check switched.persisted.hint_effort eq low \ + --check unoffered_error contains "Reasoning effort 'ultra' is not offered for the current model" \ + --check blank_error contains "effort must be a non-empty string" \ + --check after_refused.config_calls len-eq 0 --check after_refused.current_effort eq low --check after_refused.acp_effort eq low \ + --check on_m2.current_model_id eq m2 --check on_m2.current_effort eq null --check on_m2.available_efforts len-eq 0 \ + --check on_m2.acp_effort eq low --check on_m2_error contains "offers no reasoning effort setting" \ + --check back_on_m1.config_calls.0 eq model=m1 --check back_on_m1.config_calls.1 eq effort=low --check back_on_m1.current_effort eq low \ + --check next_turn_error missing --check next_turn.status eq finished --check next_turn.prompts eq 1 --check next_turn.same_session eq true \ + --check skipped_error missing --check skipped.status eq finished --check skipped.config_calls len-eq 1 \ + --check skipped.current_effort eq medium --check skipped.acp_effort eq ultra --check skipped.warned eq true \ + --check deferred.processes_started eq 0 --check deferred.acp_effort eq high --check deferred.hint_effort eq high \ + --check deferred.session_id eq null --check deferred.persisted.acp_effort eq high --check deferred.blank_error contains "non-empty" \ + --check deferred_error missing --check deferred_started.config_calls.0 eq effort=high --check deferred_started.current_effort eq high \ + --check legacy.suffix.current_model_id eq gpt-5.5 --check legacy.suffix.config_calls.1 eq reasoning_effort=high --check legacy.suffix.current_effort eq high \ + --check legacy.explicit.current_model_id eq gpt-5.5 --check legacy.explicit.config_calls.1 eq reasoning_effort=low --check legacy.explicit.current_effort eq low \ + --check native_error eq "ValueError: switch_acp_effort is only supported for ACP conversations." \ + --check discovery.error eq null --check discovery.current_model_id eq m1 --check discovery.current_effort eq medium \ + --check discovery.available_efforts len-eq 3 --check discovery.efforts_by_model.m1 len-eq 3 --check discovery.efforts_by_model.m2 len-eq 0 \ + --check settings_agent_effort eq low --check settings_roundtrip_effort eq low + ``` + What this shows: + - At session start the SDK sets the model, then the effort through the option tagged `thought_level`; `current_effort` and `available_efforts` come from the server's reply. + - A live switch is one `set_config_option(effort)` on the same session, and `acp_effort` plus the `acp_current_effort` hint land in `base_state.json`. + - A level the model does not offer, a blank level and a model with no effort option raise `ValueError` without a server call or a change. + - On `m2` the session reports no effort while `acp_effort` stays `low`; back on `m1` (whose effort the server reset to `medium`) the SDK re-applies `low`. The next turn runs in the same session. + - At start, an effort the model does not offer is skipped with a warning and the turn runs at the server default. + - Before the first run, the switch starts no subprocess; the first session applies it (no model is pinned, so only the effort is set). + - A legacy `gpt-5.5/high` id is split for codex-acp, `current_model_id` is the bare `gpt-5.5`, and an explicit `acp_effort` wins over the suffix. + - Discovery reports the default model's effort and the levels of every model. + + The server's view of this is `F09.acp-effort-deferred`, `F09.acp-effort-live`, `F09.acp-effort-unoffered` and `F09.acp-effort-model-switch` (`POST /api/conversations/{id}/switch_acp_effort` calls `switch_acp_effort`). - **Traces of an ACP turn (`S21.tracing`).** `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` points at the local collector from `programs/S20/otlp.py`. The user's message contains the secret value. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) @@ -234,7 +280,7 @@ Preconditions: --check probes.claude-code.current_in_models eq true --check probes.codex.current_in_models eq true \ --check probes.gemini-cli.current_in_models eq true --check probes.pi.current_in_models eq true ``` - Here all six providers passed in about 155 s on a cold npm cache: claude-agent-acp 0.63.0, codex-acp 1.10.0, gemini-cli 0.46.0, Kimi Code CLI 0.38.0, OpenCode 1.18.23 and pi-acp 0.0.33. + Here all six providers passed in about 90 s: claude-agent-acp 0.99.1, codex-acp 2.2.2, gemini-cli 0.46.0, Kimi Code CLI 0.38.0, OpenCode 1.18.23 and pi-acp 0.0.33 (the versions the install catalog pins). - **Example 40 with Claude Code (`S21.live-turn`, `S21.step`, `S21.ask-agent`, `S21.usage-cost`), blocked: needs `ANTHROPIC_API_KEY` (optionally `ANTHROPIC_BASE_URL` for a LiteLLM proxy), plus `npx` and network for the image URL.** Run it from the repo root. The agent lists the `openhands-sdk/openhands/sdk/agent/` files, describes the python.org image and answers the `ask_agent` question. The example prints `EXAMPLE_COST` twice, then `Done!`. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) diff --git a/.pr/bugs/2026-11-11/litellm-startup-race/server-startup-crash.log b/.pr/bugs/2026-11-11/litellm-startup-race/server-startup-crash.log new file mode 100644 index 00000000000..00c8de21c1a --- /dev/null +++ b/.pr/bugs/2026-11-11/litellm-startup-race/server-startup-crash.log @@ -0,0 +1,82 @@ + +=== 2026-11-11T08:41:33Z start: /home/user/software-agent-sdk/.venv/bin/python -m openhands.agent_server --host 127.0.0.1 --port 37231 +Exception in thread litellm-model-cost-map-retry: +Traceback (most recent call last): + File "", line 203, in _run_module_as_main + File "", line 88, in _run_code + File "/home/user/software-agent-sdk/openhands-agent-server/openhands/agent_server/__main__.py", line 13, in + from openhands.agent_server.logging_config import LOGGING_CONFIG + File "/home/user/software-agent-sdk/openhands-agent-server/openhands/agent_server/logging_config.py", line 8, in + from openhands.sdk.logger import ENV_JSON, ENV_LOG_LEVEL, IN_CI + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/__init__.py", line 5, in + from openhands.sdk.agent import ( + ...<2 lines>... + ) + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/agent/__init__.py", line 5, in + from openhands.sdk.agent.agent import Agent + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/agent/agent.py", line 12, in + import openhands.sdk.security.analyzer as analyzer + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/security/__init__.py", line 1, in + from openhands.sdk.security.analyzer import SecurityAnalysis, SecurityAnalyzerBase + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/security/analyzer.py", line 6, in + from openhands.sdk.event.base import Event + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/event/__init__.py", line 1, in + from openhands.sdk.event.acp_tool_call import ACPToolCallEvent + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/event/acp_tool_call.py", line 9, in + from openhands.sdk.event.base import Event + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/event/base.py", line 9, in + from openhands.sdk.event.types import ROOT_PARENT_ID, EventID, SourceType + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/event/types.py", line 3, in + from openhands.sdk.utils.masking import SkipSecretMasking + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/utils/__init__.py", line 3, in + from .command import sanitized_env + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/utils/command.py", line 9, in + from openhands.sdk.logger import get_logger + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/logger/__init__.py", line 1, in + from .logger import ( + ...<7 lines>... + ) + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/logger/logger.py", line 16, in + import litellm + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/__init__.py", line 563, in + model_cost = get_model_cost_map(url=model_cost_map_url) +Exception in thread litellm-model-cost-map-retry: + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/litellm_core_utils/get_model_cost_map.py", line 672, in get_model_cost_map + verbose_logger.warning( + ~~~~~~~~~~~~~~~~~~~~~~^ + "LiteLLM: Failed to fetch remote model cost map from %s: %s. Falling back to local backup.", + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + url, + ^^^^ + outcome.reason, + ^^^^^^^^^^^^^^^ + ) + ^ + File "/usr/local/lib/python3.13/logging/__init__.py", line 1537, in warning + self._log(WARNING, msg, args, **kwargs) + ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/usr/local/lib/python3.13/logging/__init__.py", line 1670, in _log + self.handle(record) + ~~~~~~~~~~~^^^^^^^^ + File "/usr/local/lib/python3.13/logging/__init__.py", line 1681, in handle + maybe_record = self.filter(record) + File "/usr/local/lib/python3.13/logging/__init__.py", line 867, in filter + result = f.filter(record) + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/_logging.py", line 551, in filter + return _process_record( + record, + ...<2 lines>... + redact=_ENABLE_SECRET_REDACTION, + ) + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/_logging.py", line 455, in _process_record + candidate: Final = diagnostics.run( + ~~~~~~~~~~~~~~~^ + lambda native: native.process_diagnostic(message, exception, stack, leaves, (redact, base64_limit, text_limit)), + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + lambda: _python_process_diagnostic(message, exception, stack, leaves, redact, base64_limit, text_limit), + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + ) + ^ + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/rust_bridge/diagnostics.py", line 47, in run + from litellm.rust_bridge.catalog import LoggerContext, decision +ImportError: cannot import name 'LoggerContext' from partially initialized module 'litellm.rust_bridge.catalog' (most likely due to a circular import) (/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/rust_bridge/catalog.py) diff --git a/.pr/bugs/2026-11-11/litellm-startup-race/server-startup-deadlock.log b/.pr/bugs/2026-11-11/litellm-startup-race/server-startup-deadlock.log new file mode 100644 index 00000000000..cfd889d28c3 --- /dev/null +++ b/.pr/bugs/2026-11-11/litellm-startup-race/server-startup-deadlock.log @@ -0,0 +1,82 @@ +=== 2026-11-11T08:34:24Z start: /home/user/software-agent-sdk/.venv/bin/python -m openhands.agent_server --host 127.0.0.1 --port 49719 +Traceback (most recent call last): + File "", line 203, in _run_module_as_main + File "", line 88, in _run_code + File "/home/user/software-agent-sdk/openhands-agent-server/openhands/agent_server/__main__.py", line 13, in + from openhands.agent_server.logging_config import LOGGING_CONFIG + File "/home/user/software-agent-sdk/openhands-agent-server/openhands/agent_server/logging_config.py", line 8, in + from openhands.sdk.logger import ENV_JSON, ENV_LOG_LEVEL, IN_CI + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/__init__.py", line 5, in + from openhands.sdk.agent import ( + ...<2 lines>... + ) + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/agent/__init__.py", line 5, in + from openhands.sdk.agent.agent import Agent + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/agent/agent.py", line 12, in + import openhands.sdk.security.analyzer as analyzer + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/security/__init__.py", line 1, in + from openhands.sdk.security.analyzer import SecurityAnalysis, SecurityAnalyzerBase + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/security/analyzer.py", line 6, in + from openhands.sdk.event.base import Event + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/event/__init__.py", line 1, in + from openhands.sdk.event.acp_tool_call import ACPToolCallEvent + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/event/acp_tool_call.py", line 9, in + from openhands.sdk.event.base import Event + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/event/base.py", line 9, in + from openhands.sdk.event.types import ROOT_PARENT_ID, EventID, SourceType + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/event/types.py", line 3, in + from openhands.sdk.utils.masking import SkipSecretMasking + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/utils/__init__.py", line 3, in + from .command import sanitized_env + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/utils/command.py", line 9, in + from openhands.sdk.logger import get_logger + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/logger/__init__.py", line 1, in + from .logger import ( + ...<7 lines>... + ) + File "/home/user/software-agent-sdk/openhands-sdk/openhands/sdk/logger/logger.py", line 16, in + import litellm + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/__init__.py", line 563, in + model_cost = get_model_cost_map(url=model_cost_map_url) + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/litellm_core_utils/get_model_cost_map.py", line 672, in get_model_cost_map + verbose_logger.warning( + ~~~~~~~~~~~~~~~~~~~~~~^ + "LiteLLM: Failed to fetch remote model cost map from %s: %s. Falling back to local backup.", + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + url, + ^^^^ + outcome.reason, + ^^^^^^^^^^^^^^^ + ) + ^ + File "/usr/local/lib/python3.13/logging/__init__.py", line 1537, in warning + self._log(WARNING, msg, args, **kwargs) + ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/usr/local/lib/python3.13/logging/__init__.py", line 1670, in _log + self.handle(record) + ~~~~~~~~~~~^^^^^^^^ + File "/usr/local/lib/python3.13/logging/__init__.py", line 1681, in handle + maybe_record = self.filter(record) + File "/usr/local/lib/python3.13/logging/__init__.py", line 867, in filter + result = f.filter(record) + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/_logging.py", line 551, in filter + return _process_record( + record, + ...<2 lines>... + redact=_ENABLE_SECRET_REDACTION, + ) + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/_logging.py", line 455, in _process_record + candidate: Final = diagnostics.run( + ~~~~~~~~~~~~~~~^ + lambda native: native.process_diagnostic(message, exception, stack, leaves, (redact, base64_limit, text_limit)), + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + lambda: _python_process_diagnostic(message, exception, stack, leaves, redact, base64_limit, text_limit), + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + ) + ^ + File "/home/user/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/rust_bridge/diagnostics.py", line 47, in run + from litellm.rust_bridge.catalog import LoggerContext, decision + File "", line 1366, in _find_and_load + File "", line 418, in __enter__ + File "", line 346, in acquire +_frozen_importlib._DeadlockError: deadlock detected by _ModuleLock('litellm.rust_bridge.catalog') at 140349952615248 diff --git a/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/out-reserved-3.txt b/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/out-reserved-3.txt new file mode 100644 index 00000000000..98e103652a5 --- /dev/null +++ b/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/out-reserved-3.txt @@ -0,0 +1,32 @@ ++----------------------------------------------------------------------+ +| OpenHands SDK v1.61.0 | +| | +| Report a bug: github.com/OpenHands/software-agent-sdk/issues | +| Get help: openhands.dev/joinslack | +| Scale up: openhands.dev/product/sdk | +| | +| Set OPENHANDS_SUPPRESS_BANNER=1 to hide this message | ++----------------------------------------------------------------------+ + +[11/11/26 09:06:20] WARNING Duplicate tool name registered: registry.py:154 + terminal +[11/11/26 09:06:20] WARNING No persistence_dir provided; falling state.py:562 + back to InMemoryFileStore. EventLog + data will not persist across requests. +[11/11/26 09:06:20] INFO Created new conversation state.py:645 + fca4a5dc-22f8-4a39-aa5b-a937552d00ef +[11/11/26 09:06:21] INFO TmuxPanePool initialized: tmux_pane_pool.py:149 + session=openhands-pool-None-e + 7a12e26-c6c6-407d-8fd5-ab6612 + 227959, max_panes=4 +[11/11/26 09:06:21] INFO TerminalExecutor initialized (pool impl.py:148 + mode) working_dir: + /tmp/tmuxleak-ws-50sh2u8k, username: + None, max_panes: 4 +run: raised ValueError: response_schema fields ['summary'] are reserved + traceback mentions set_response_schema: True + traceback mentions resolve_tool: True +close: done + socket .openhands-socket-creation.lock: rc=1 no server running on /tmp/tl-TXzEdI/tmux-0/.openhands-socket-creation.lock + socket openhands-7615099345eb46b786fa31ddcdeeb622: rc=0 openhands-pool-None-e7a12e26-c6c6-407d-8fd5-ab6612227959: 1 windows (created Wed Nov 11 09:06:20 2026) +RESULT mode=reserved tmux_servers_alive_after_close=1 ['openhands-7615099345eb46b786fa31ddcdeeb622'] diff --git a/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/out-reserved-b1.txt b/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/out-reserved-b1.txt new file mode 100644 index 00000000000..d49951fd177 --- /dev/null +++ b/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/out-reserved-b1.txt @@ -0,0 +1,32 @@ ++----------------------------------------------------------------------+ +| OpenHands SDK v1.60.1 | +| | +| Report a bug: github.com/OpenHands/software-agent-sdk/issues | +| Get help: openhands.dev/joinslack | +| Scale up: openhands.dev/product/sdk | +| | +| Set OPENHANDS_SUPPRESS_BANNER=1 to hide this message | ++----------------------------------------------------------------------+ + +[11/11/26 09:07:26] WARNING Duplicate tool name registered: registry.py:154 + terminal +[11/11/26 09:07:26] WARNING No persistence_dir provided; falling state.py:562 + back to InMemoryFileStore. EventLog + data will not persist across requests. +[11/11/26 09:07:26] INFO Created new conversation state.py:645 + ba3b224b-8d7a-45ee-b5bc-542953ecaf60 +[11/11/26 09:07:27] INFO TmuxPanePool initialized: tmux_pane_pool.py:149 + session=openhands-pool-None-0 + 9d300b7-dc3b-4130-a4c9-6be00b + e697a4, max_panes=4 +[11/11/26 09:07:27] INFO TerminalExecutor initialized (pool impl.py:145 + mode) working_dir: + /tmp/tmuxleak-ws-tgd6iof1, username: + None, max_panes: 4 +run: raised ValueError: response_schema fields ['summary'] are reserved + traceback mentions set_response_schema: True + traceback mentions resolve_tool: True +close: done + socket .openhands-socket-creation.lock: rc=1 no server running on /tmp/tl-VT0vZj/tmux-0/.openhands-socket-creation.lock + socket openhands-c2e33c357bec4c2181b8f9c95612d2ff: rc=0 openhands-pool-None-09d300b7-dc3b-4130-a4c9-6be00be697a4: 1 windows (created Wed Nov 11 09:07:26 2026) +RESULT mode=reserved tmux_servers_alive_after_close=1 ['openhands-c2e33c357bec4c2181b8f9c95612d2ff'] diff --git a/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/out-valid-3.txt b/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/out-valid-3.txt new file mode 100644 index 00000000000..8f62d2987ac --- /dev/null +++ b/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/out-valid-3.txt @@ -0,0 +1,33 @@ ++----------------------------------------------------------------------+ +| OpenHands SDK v1.61.0 | +| | +| Report a bug: github.com/OpenHands/software-agent-sdk/issues | +| Get help: openhands.dev/joinslack | +| Scale up: openhands.dev/product/sdk | +| | +| Set OPENHANDS_SUPPRESS_BANNER=1 to hide this message | ++----------------------------------------------------------------------+ + +[11/11/26 09:06:28] WARNING Duplicate tool name registered: registry.py:154 + terminal +[11/11/26 09:06:28] WARNING No persistence_dir provided; falling state.py:562 + back to InMemoryFileStore. EventLog + data will not persist across requests. +[11/11/26 09:06:28] INFO Created new conversation state.py:645 + d9d67c7b-912f-4061-83ef-1455e115127b +[11/11/26 09:06:29] INFO TmuxPanePool initialized: tmux_pane_pool.py:149 + session=openhands-pool-None-c + d7f58c7-5c33-442e-94c1-dc5183 + 85c170, max_panes=4 +[11/11/26 09:06:29] INFO TerminalExecutor initialized (pool impl.py:148 + mode) working_dir: + /tmp/tmuxleak-ws-9jxt11b9, username: + None, max_panes: 4 +[11/11/26 09:06:29] INFO Loaded 1 tools from spec base.py:585 +run: raised ConversationRunError: Conversation run failed for id=d9d67c7b-912f-4061-83ef-1455e115127b: TestLLM: no more scripted responses (exhausted after 0 calls) + traceback mentions set_response_schema: False + traceback mentions resolve_tool: False +close: done + socket .openhands-socket-creation.lock: rc=1 no server running on /tmp/tl-segEVU/tmux-0/.openhands-socket-creation.lock + socket openhands-929b8ac6294b460bbc9347ef5d1839d1: rc=1 no server running on /tmp/tl-segEVU/tmux-0/openhands-929b8ac6294b460bbc9347ef5d1839d1 +RESULT mode=valid tmux_servers_alive_after_close=0 [] diff --git a/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/tmux_leak.py b/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/tmux_leak.py new file mode 100644 index 00000000000..4c69f1d46eb --- /dev/null +++ b/.pr/bugs/2026-11-11/rejected-schema-tmux-leak/tmux_leak.py @@ -0,0 +1,63 @@ +"""Does a rejected TerminalTool response_schema leave a tmux server after close()? + +Usage: TMUX_TMPDIR= python tmux_leak.py reserved|valid +""" +import os +import subprocess +import sys +import tempfile +import traceback +from pathlib import Path + +from pydantic import BaseModel + +from openhands.sdk import Agent, Conversation, Tool +from openhands.sdk.testing import TestLLM +from openhands.sdk.tool import register_tool +from openhands.tools.terminal import TerminalTool + + +class Reserved(BaseModel): + summary: str + + +class Valid(BaseModel): + verdict: str + + +def live_servers() -> list[str]: + d = Path(os.environ["TMUX_TMPDIR"]) / f"tmux-{os.getuid()}" + alive = [] + for sock in sorted(d.glob("*")) if d.exists() else []: + r = subprocess.run(["tmux", "-L", sock.name, "ls"], capture_output=True, text=True) + print(f" socket {sock.name}: rc={r.returncode} {(r.stdout or r.stderr).strip()[:120]}") + if r.returncode == 0: + alive.append(sock.name) + return alive + + +def main(mode: str) -> None: + schema = Reserved if mode == "reserved" else Valid + register_tool(TerminalTool.name, TerminalTool) + llm = TestLLM.from_messages([]) if hasattr(TestLLM, "from_messages") else TestLLM() + agent = Agent(llm=llm, tools=[Tool(name=TerminalTool.name, params={"response_schema": schema})]) + ws = tempfile.mkdtemp(prefix="tmuxleak-ws-") + conv = Conversation(agent=agent, workspace=ws, visualizer=None) + try: + conv.send_message("x") + conv.run() + print("run: returned normally") + except Exception as e: # noqa: BLE001 + print(f"run: raised {type(e).__name__}: {e}") + tb = traceback.format_exc() + for marker in ("set_response_schema", "resolve_tool"): + print(f" traceback mentions {marker}: {marker in tb}") + finally: + conv.close() + print("close: done") + alive = live_servers() + print(f"RESULT mode={mode} tmux_servers_alive_after_close={len(alive)} {alive}") + + +if __name__ == "__main__": + main(sys.argv[1]) diff --git a/.pr/bugs/2026-11-11/resume-missing-tool-module/map-run-pre6669.json b/.pr/bugs/2026-11-11/resume-missing-tool-module/map-run-pre6669.json new file mode 100644 index 00000000000..4cec0c4601f --- /dev/null +++ b/.pr/bugs/2026-11-11/resume-missing-tool-module/map-run-pre6669.json @@ -0,0 +1,62 @@ +{ + "ok": false, + "file": "F27-hooks-subagents-tools.md", + "summary": { + "pass": 2, + "fail": 1, + "xfail": 0, + "xpass": 0, + "blocked": 0, + "skipped": 0, + "no-commands": 0 + }, + "bullets": [ + { + "bullet": "Preconditions", + "ids": [], + "line": 147, + "status": "pass", + "exit_code": 0, + "elapsed_ms": 20870, + "transcript": "/agent-server/20261111-090117-7a26-map-f27/evidence/map-run/F27-hooks-subagents-tools-00-preconditions.json" + }, + { + "bullet": "Custom tool module (`F27.custom-tool-modules`).", + "ids": [ + "F27.custom-tool-modules" + ], + "line": 687, + "status": "pass", + "exit_code": 0, + "elapsed_ms": 51808, + "transcript": "/agent-server/20261111-090117-7a26-map-f27/evidence/map-run/F27-hooks-subagents-tools-01-custom-tool-module-f27-custom-tool-modul.json" + }, + { + "bullet": "Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug.", + "ids": [ + "F27.custom-tool-missing-on-resume" + ], + "line": 777, + "status": "fail", + "exit_code": 1, + "elapsed_ms": 16361, + "transcript": "/agent-server/20261111-090117-7a26-map-f27/evidence/map-run/F27-hooks-subagents-tools-02-tool-module-gone-on-resume-f27-custom-to.json", + "failed_line": 4, + "failed_command": "control-agent-server api POST \"/api/conversations/$MCID/events\" --json '{\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"Reply with the single word ok.\"}], \"run\": false}' --expect 2xx,4xx --save F27.custom-tool-missing-on-resume/send", + "stdout_tail": "nown kind 'QaF27EchoObservation' for openhands.sdk.tool.schema.Observation; Expected one of: ['MCPToolObservation', 'ClassifyAndSwitchLLMObservation', 'FinishObservation', 'InvokeSkillObservation', 'SwitchLLMObservation', 'ThinkObservation', 'VisionInspectObservation', 'ClientToolObservation', 'AskOracleObservation', 'BrowserObservation', 'StartChildConversationObservation', 'DelegateObservation', 'FileEditorObservation', 'EditObservation', 'ListDirectoryObservation', 'ReadFileObservation', 'WriteFileObservation', 'GlobObservation', 'GrepObservation', 'PlanningFileEditorObservation', 'TaskObservation', 'TaskTrackerObservation', 'TerminalObservation', 'WorkflowObservation'] [type=value_error, input_value={'content': [{'cache_prom... 'QaF27EchoObservation'}, input_type=dict]\\n For further information visit https://errors.pydantic.dev/2.13/v/value_error\",\n \"error_id\": \"0f2b201e1bfd49a4957d7357ab125435\"\n },\n \"truncated\": false\n },\n \"elapsed_ms\": 1987,\n \"saved\": \"/agent-server/20261111-090117-7a26-map-f27/evidence/F27.custom-tool-missing-on-resume/send.json\",\n \"hint\": \"expected 2xx,4xx, got 500\"\n}\n", + "stderr_tail": "", + "bug_predicate": "status eq 500 && response.body.exception contains \"ToolDefinition 'qa_f27_echo' is not registered\"", + "failed_status": 1, + "observed": { + "status": 500, + "hint": "expected 2xx,4xx, got 500" + }, + "hint": "the `# bug` command failed, but not with the documented failure (`# bug: status eq 500 && response.body.exception contains \"ToolDefinition 'qa_f27_echo' is not registered\"`); the bug was not reproduced" + } + ], + "run": "/agent-server/20261111-090117-7a26-map-f27", + "launch_flags": [ + "--checkout", + "/repro/wt-pre6669" + ] +} diff --git a/.pr/bugs/2026-11-11/resume-missing-tool-module/map-run-repeat2.json b/.pr/bugs/2026-11-11/resume-missing-tool-module/map-run-repeat2.json new file mode 100644 index 00000000000..5a9380c4d57 --- /dev/null +++ b/.pr/bugs/2026-11-11/resume-missing-tool-module/map-run-repeat2.json @@ -0,0 +1,43 @@ +{ + "ok": true, + "file": "F27-hooks-subagents-tools.md", + "launch_flags": [], + "attempts": [ + { + "run": "/agent-server/20261111-090008-47b9-map-f27", + "ok": true, + "summary": { + "pass": 2, + "fail": 0, + "xfail": 1, + "xpass": 0, + "blocked": 0, + "skipped": 0, + "no-commands": 0 + }, + "statuses": { + "Preconditions": "pass", + "Custom tool module (`F27.custom-tool-modules`).": "pass", + "Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug.": "xfail" + } + }, + { + "run": "/agent-server/20261111-090141-d9c4-map-f27", + "ok": true, + "summary": { + "pass": 2, + "fail": 0, + "xfail": 1, + "xpass": 0, + "blocked": 0, + "skipped": 0, + "no-commands": 0 + }, + "statuses": { + "Preconditions": "pass", + "Custom tool module (`F27.custom-tool-modules`).": "pass", + "Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug.": "xfail" + } + } + ] +} diff --git a/.pr/bugs/2026-11-11/resume-missing-tool-module/send-500.json b/.pr/bugs/2026-11-11/resume-missing-tool-module/send-500.json new file mode 100644 index 00000000000..b3d719ceca5 --- /dev/null +++ b/.pr/bugs/2026-11-11/resume-missing-tool-module/send-500.json @@ -0,0 +1,33 @@ +{ + "request": { + "method": "POST", + "path": "/api/conversations/a9fad52f-5648-4393-8375-bdefd067db34/events", + "query": [], + "auth": "header", + "body": { + "role": "user", + "content": [ + { + "type": "text", + "text": "Reply with the single word ok." + } + ], + "run": false + }, + "files": [] + }, + "response": { + "status": 500, + "headers": { + "content-length": "145", + "content-type": "application/json" + }, + "body": { + "detail": "Internal Server Error", + "exception": "\"ToolDefinition 'qa_f27_echo' is not registered\"", + "error_id": "7b8917c293bf4d01b98150f5504613e4" + }, + "truncated": false + }, + "elapsed_ms": 1892 +} diff --git a/.pr/bugs/2026-11-11/session-socket-sync-first/independent-main-error-before-sync.json b/.pr/bugs/2026-11-11/session-socket-sync-first/independent-main-error-before-sync.json new file mode 100644 index 00000000000..884c44a75f2 --- /dev/null +++ b/.pr/bugs/2026-11-11/session-socket-sync-first/independent-main-error-before-sync.json @@ -0,0 +1,56 @@ +{ + "summary": { + "opened": true, + "frames": 2, + "kinds": { + "error": 1, + "sync": 1 + }, + "stopped": "until-kind sync", + "close": null + }, + "frames": [ + { + "direction": "opened", + "at": "2026-11-11T08:56:27Z" + }, + { + "direction": "sent", + "at": "2026-11-11T08:56:27Z", + "frame": { + "role": "assistant", + "content": [ + { + "type": "text", + "text": "x" + } + ] + } + }, + { + "direction": "received", + "at": "2026-11-11T08:56:28Z", + "kind": "error", + "frame": { + "type": "error", + "code": "AssertionError", + "detail": "Only user messages are allowed to be sent to the agent." + } + }, + { + "direction": "received", + "at": "2026-11-11T08:57:15Z", + "kind": "sync", + "frame": { + "type": "sync", + "through_seq": 9 + } + }, + { + "direction": "ended", + "at": "2026-11-11T08:57:15Z", + "stopped": "until-kind sync", + "close": null + } + ] +} diff --git a/.pr/bugs/2026-11-11/session-socket-sync-first/independent-main-valid-message-saved.json b/.pr/bugs/2026-11-11/session-socket-sync-first/independent-main-valid-message-saved.json new file mode 100644 index 00000000000..8446cfbd303 --- /dev/null +++ b/.pr/bugs/2026-11-11/session-socket-sync-first/independent-main-valid-message-saved.json @@ -0,0 +1,24 @@ +{ + "events": [ + { + "id": "05c99a8b-1ef4-408d-b4bb-4fd7e4cbcbc0", + "timestamp": "2026-11-11T08:57:15.040389", + "source": "user", + "parent_id": "8a638099-bda7-411c-a39d-15cf436a8528", + "llm_message": { + "role": "user", + "content": [ + { + "cache_prompt": false, + "type": "text", + "text": "QA_VALID_DURING_STEP_main2" + } + ], + "thinking_blocks": [] + }, + "activated_skills": [], + "extended_content": [], + "kind": "MessageEvent" + } + ] +} diff --git a/.pr/bugs/2026-11-11/session-socket-sync-first/main-error-before-sync.json b/.pr/bugs/2026-11-11/session-socket-sync-first/main-error-before-sync.json new file mode 100644 index 00000000000..226672365b1 --- /dev/null +++ b/.pr/bugs/2026-11-11/session-socket-sync-first/main-error-before-sync.json @@ -0,0 +1,704 @@ +{ + "summary": { + "opened": true, + "frames": 8, + "kinds": { + "error": 1, + "sync": 1, + "transient:ConversationStateUpdateEvent": 2, + "durable:ObservationEvent": 1, + "durable:ConversationStateUpdateEvent": 2, + "durable:MessageEvent": 1 + }, + "stopped": "duration", + "close": null + }, + "frames": [ + { + "direction": "opened", + "at": "2026-11-11T08:46:03Z" + }, + { + "direction": "sent", + "at": "2026-11-11T08:46:03Z", + "frame": { + "role": "assistant", + "content": [ + { + "type": "text", + "text": "QA_F07_RACE" + } + ] + } + }, + { + "direction": "received", + "at": "2026-11-11T08:46:03Z", + "kind": "error", + "frame": { + "type": "error", + "code": "AssertionError", + "detail": "Only user messages are allowed to be sent to the agent." + } + }, + { + "direction": "received", + "at": "2026-11-11T08:46:28Z", + "kind": "sync", + "frame": { + "type": "sync", + "through_seq": 7 + } + }, + { + "direction": "received", + "at": "2026-11-11T08:46:28Z", + "kind": "transient:ConversationStateUpdateEvent", + "frame": { + "type": "transient", + "event": { + "id": "baeec543-a9ae-4fb1-b4e6-c0657cd54eb7", + "timestamp": "2026-11-11T08:46:28.229992", + "source": "environment", + "key": "full_state", + "value": { + "id": "0541087f-3877-441d-91a5-1cb78f079ed0", + "agent": { + "llm": { + "model": "deepseek/deepseek-flash", + "api_key": "**********", + "auth_type": "api_key", + "openrouter_site_url": "https://docs.all-hands.dev/", + "openrouter_app_name": "OpenHands", + "num_retries": 5, + "retry_multiplier": 8.0, + "retry_min_wait": 8, + "retry_max_wait": 64, + "timeout": 300, + "stream_idle_timeout": 300.0, + "max_message_chars": 30000, + "api_mode": "auto", + "capability_overrides": {}, + "stream": false, + "drop_params": true, + "disable_stop_word": false, + "caching_prompt": true, + "log_completions": false, + "log_completions_folder": "logs/completions", + "native_tool_calling": true, + "reasoning_effort": "high", + "enable_encrypted_reasoning": true, + "prompt_cache_retention": "24h", + "extended_thinking_budget": 200000, + "usage_id": "default", + "litellm_extra_body": {}, + "is_subscription": false + }, + "tools": [ + { + "name": "terminal", + "params": {} + }, + { + "name": "file_editor", + "params": {} + }, + { + "name": "task_tracker", + "params": {} + }, + { + "name": "browser_tool_set", + "params": {} + } + ], + "mcp_config": {}, + "include_default_tools": [ + "FinishTool", + "SwitchLLMTool" + ], + "agent_context": { + "skills": [], + "load_user_skills": false, + "load_public_skills": false, + "marketplace_path": "marketplaces/default.json", + "registered_marketplaces": [], + "load_project_skills": false, + "load_memory": false, + "disabled_skills": [], + "current_datetime": "2026-11-11T08:46:00.216692Z" + }, + "system_prompt_filename": "system_prompt.j2", + "security_policy_filename": "security_policy.j2", + "system_prompt_kwargs": { + "llm_security_analyzer": true + }, + "condenser": { + "llm": { + "model": "deepseek/deepseek-flash", + "api_key": "**********", + "auth_type": "api_key", + "openrouter_site_url": "https://docs.all-hands.dev/", + "openrouter_app_name": "OpenHands", + "num_retries": 5, + "retry_multiplier": 8.0, + "retry_min_wait": 8, + "retry_max_wait": 64, + "timeout": 300, + "stream_idle_timeout": 300.0, + "max_message_chars": 30000, + "api_mode": "auto", + "capability_overrides": {}, + "stream": false, + "drop_params": true, + "disable_stop_word": false, + "caching_prompt": true, + "log_completions": false, + "log_completions_folder": "logs/completions", + "native_tool_calling": true, + "reasoning_effort": "high", + "enable_encrypted_reasoning": true, + "prompt_cache_retention": "24h", + "extended_thinking_budget": 200000, + "usage_id": "condenser", + "litellm_extra_body": {}, + "is_subscription": false + }, + "max_size": 240, + "keep_first": 2, + "minimum_progress": 0.1, + "hard_context_reset_max_retries": 5, + "hard_context_reset_context_scaling": 0.8, + "kind": "LLMSummarizingCondenser" + }, + "tool_concurrency_limit": 1, + "kind": "Agent" + }, + "workspace": { + "working_dir": "/agent-server/20261111-084544-9aca/fixtures/workspaces/084600-202059", + "kind": "LocalWorkspace" + }, + "persistence_dir": "/agent-server/20261111-084544-9aca/server/workspace/conversations/0541087f3877441d91a51cb78f079ed0", + "max_iterations": 500, + "stuck_detection": true, + "execution_status": "running", + "confirmation_policy": { + "kind": "NeverConfirm" + }, + "activated_knowledge_skills": [], + "activated_path_rules": [], + "invoked_skills": [], + "blocked_actions": {}, + "blocked_messages": {}, + "last_user_message_id": "35b8053e-8324-4656-a36c-5e0a117c9aac", + "leaf_event_id": "527650f7-1612-42c0-858f-88eac726c250", + "head_is_empty": false, + "stats": { + "usage_to_metrics": { + "default": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.002433468, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 10265, + "completion_tokens": 277, + "cache_read_tokens": 3328, + "cache_write_tokens": 0, + "reasoning_tokens": 163, + "context_window": 1000000, + "per_turn_token": 10252, + "response_id": "" + }, + "costs": [ + { + "model": "deepseek/deepseek-flash", + "cost": 0.00022199999999999998, + "timestamp": 1794386762.283774 + }, + { + "model": "deepseek/deepseek-flash", + "cost": 0.002211468, + "timestamp": 1794386762.5464773 + } + ], + "response_latencies": [ + { + "model": "deepseek/deepseek-flash", + "latency": 1.625413417816162, + "response_id": "5525774b-ccf4-4d3b-ac44-049a0f48894d" + }, + { + "model": "deepseek/deepseek-flash", + "latency": 1.8880395889282227, + "response_id": "fde20ca6-dfd3-4a62-8d4b-ea0d24d3b24f" + } + ], + "token_usages": [ + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 140, + "completion_tokens": 150, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 140, + "context_window": 1000000, + "per_turn_token": 290, + "response_id": "5525774b-ccf4-4d3b-ac44-049a0f48894d" + }, + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 10125, + "completion_tokens": 127, + "cache_read_tokens": 3328, + "cache_write_tokens": 0, + "reasoning_tokens": 23, + "context_window": 1000000, + "per_turn_token": 10252, + "response_id": "fde20ca6-dfd3-4a62-8d4b-ea0d24d3b24f" + } + ] + }, + "condenser": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.0, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 0, + "completion_tokens": 0, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 0, + "context_window": 0, + "per_turn_token": 0, + "response_id": "" + }, + "costs": [], + "response_latencies": [], + "token_usages": [] + } + } + }, + "secret_registry": { + "secret_sources": {} + }, + "tags": {}, + "agent_state": {} + }, + "kind": "ConversationStateUpdateEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:46:28Z", + "kind": "durable:ObservationEvent", + "frame": { + "type": "durable", + "seq": 7, + "event": { + "id": "527650f7-1612-42c0-858f-88eac726c250", + "timestamp": "2026-11-11T08:46:28.223056", + "source": "environment", + "parent_id": "3da8f685-1431-414e-9d5f-d8d395227021", + "tool_name": "terminal", + "tool_call_id": "call_00_GdMTRyVpab2jknnoE4Et4869", + "observation": { + "content": [ + { + "cache_prompt": false, + "type": "text", + "text": "QA_DONE" + } + ], + "is_error": false, + "command": "sleep 25 && echo QA_DONE", + "exit_code": 0, + "timeout": false, + "metadata": { + "exit_code": 0, + "pid": -1, + "username": "root", + "hostname": "vm", + "working_dir": "/agent-server/20261111-084544-9aca/fixtures/workspaces/084600-202059", + "py_interpreter_path": "/usr/local/bin/python", + "prefix": "", + "suffix": "\n[The command completed with exit code 0.]" + }, + "full_output_save_dir": "/agent-server/20261111-084544-9aca/server/workspace/conversations/0541087f3877441d91a51cb78f079ed0/observations", + "kind": "TerminalObservation" + }, + "action_id": "3da8f685-1431-414e-9d5f-d8d395227021", + "extended_content": [], + "kind": "ObservationEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:46:30Z", + "kind": "durable:ConversationStateUpdateEvent", + "frame": { + "type": "durable", + "seq": 8, + "event": { + "id": "e5fa1cbe-c00f-492f-a59e-ae5598acc5ba", + "timestamp": "2026-11-11T08:46:30.326098", + "source": "environment", + "parent_id": "527650f7-1612-42c0-858f-88eac726c250", + "key": "stats", + "value": { + "usage_to_metrics": { + "default": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.00258504, + "max_budget_per_task": null, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 20608, + "completion_tokens": 295, + "cache_read_tokens": 13440, + "cache_write_tokens": 0, + "reasoning_tokens": 163, + "context_window": 1000000, + "per_turn_token": 10361, + "response_id": "" + } + }, + "condenser": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.0, + "max_budget_per_task": null, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 0, + "completion_tokens": 0, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 0, + "context_window": 0, + "per_turn_token": 0, + "response_id": "" + } + } + } + }, + "kind": "ConversationStateUpdateEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:46:30Z", + "kind": "durable:MessageEvent", + "frame": { + "type": "durable", + "seq": 9, + "event": { + "id": "17e3a720-93c0-4a38-b2bc-a87eaf7533ac", + "timestamp": "2026-11-11T08:46:30.329900", + "source": "agent", + "parent_id": "527650f7-1612-42c0-858f-88eac726c250", + "llm_message": { + "role": "assistant", + "content": [ + { + "cache_prompt": false, + "type": "text", + "text": "Command ran successfully \u2014 output: `QA_DONE` (exit code 0)." + } + ], + "reasoning_content": "", + "thinking_blocks": [] + }, + "llm_response_id": "b308785c-814f-49fb-8bd7-ee7da763497d", + "activated_skills": [], + "extended_content": [], + "kind": "MessageEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:46:30Z", + "kind": "durable:ConversationStateUpdateEvent", + "frame": { + "type": "durable", + "seq": 10, + "event": { + "id": "5095988e-6451-46eb-ba6a-27a937b8606b", + "timestamp": "2026-11-11T08:46:30.332412", + "source": "environment", + "parent_id": "17e3a720-93c0-4a38-b2bc-a87eaf7533ac", + "key": "execution_status", + "value": "finished", + "kind": "ConversationStateUpdateEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:46:30Z", + "kind": "transient:ConversationStateUpdateEvent", + "frame": { + "type": "transient", + "event": { + "id": "73b5c73e-4271-48b7-98db-cc40bd852ebf", + "timestamp": "2026-11-11T08:46:30.341124", + "source": "environment", + "key": "full_state", + "value": { + "id": "0541087f-3877-441d-91a5-1cb78f079ed0", + "agent": { + "llm": { + "model": "deepseek/deepseek-flash", + "api_key": "**********", + "auth_type": "api_key", + "openrouter_site_url": "https://docs.all-hands.dev/", + "openrouter_app_name": "OpenHands", + "num_retries": 5, + "retry_multiplier": 8.0, + "retry_min_wait": 8, + "retry_max_wait": 64, + "timeout": 300, + "stream_idle_timeout": 300.0, + "max_message_chars": 30000, + "api_mode": "auto", + "capability_overrides": {}, + "stream": false, + "drop_params": true, + "disable_stop_word": false, + "caching_prompt": true, + "log_completions": false, + "log_completions_folder": "logs/completions", + "native_tool_calling": true, + "reasoning_effort": "high", + "enable_encrypted_reasoning": true, + "prompt_cache_retention": "24h", + "extended_thinking_budget": 200000, + "usage_id": "default", + "litellm_extra_body": {}, + "is_subscription": false + }, + "tools": [ + { + "name": "terminal", + "params": {} + }, + { + "name": "file_editor", + "params": {} + }, + { + "name": "task_tracker", + "params": {} + }, + { + "name": "browser_tool_set", + "params": {} + } + ], + "mcp_config": {}, + "include_default_tools": [ + "FinishTool", + "SwitchLLMTool" + ], + "agent_context": { + "skills": [], + "load_user_skills": false, + "load_public_skills": false, + "marketplace_path": "marketplaces/default.json", + "registered_marketplaces": [], + "load_project_skills": false, + "load_memory": false, + "disabled_skills": [], + "current_datetime": "2026-11-11T08:46:00.216692Z" + }, + "system_prompt_filename": "system_prompt.j2", + "security_policy_filename": "security_policy.j2", + "system_prompt_kwargs": { + "llm_security_analyzer": true + }, + "condenser": { + "llm": { + "model": "deepseek/deepseek-flash", + "api_key": "**********", + "auth_type": "api_key", + "openrouter_site_url": "https://docs.all-hands.dev/", + "openrouter_app_name": "OpenHands", + "num_retries": 5, + "retry_multiplier": 8.0, + "retry_min_wait": 8, + "retry_max_wait": 64, + "timeout": 300, + "stream_idle_timeout": 300.0, + "max_message_chars": 30000, + "api_mode": "auto", + "capability_overrides": {}, + "stream": false, + "drop_params": true, + "disable_stop_word": false, + "caching_prompt": true, + "log_completions": false, + "log_completions_folder": "logs/completions", + "native_tool_calling": true, + "reasoning_effort": "high", + "enable_encrypted_reasoning": true, + "prompt_cache_retention": "24h", + "extended_thinking_budget": 200000, + "usage_id": "condenser", + "litellm_extra_body": {}, + "is_subscription": false + }, + "max_size": 240, + "keep_first": 2, + "minimum_progress": 0.1, + "hard_context_reset_max_retries": 5, + "hard_context_reset_context_scaling": 0.8, + "kind": "LLMSummarizingCondenser" + }, + "tool_concurrency_limit": 1, + "kind": "Agent" + }, + "workspace": { + "working_dir": "/agent-server/20261111-084544-9aca/fixtures/workspaces/084600-202059", + "kind": "LocalWorkspace" + }, + "persistence_dir": "/agent-server/20261111-084544-9aca/server/workspace/conversations/0541087f3877441d91a51cb78f079ed0", + "max_iterations": 500, + "stuck_detection": true, + "execution_status": "finished", + "confirmation_policy": { + "kind": "NeverConfirm" + }, + "activated_knowledge_skills": [], + "activated_path_rules": [], + "invoked_skills": [], + "blocked_actions": {}, + "blocked_messages": {}, + "last_user_message_id": "35b8053e-8324-4656-a36c-5e0a117c9aac", + "leaf_event_id": "17e3a720-93c0-4a38-b2bc-a87eaf7533ac", + "head_is_empty": false, + "stats": { + "usage_to_metrics": { + "default": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.00258504, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 20608, + "completion_tokens": 295, + "cache_read_tokens": 13440, + "cache_write_tokens": 0, + "reasoning_tokens": 163, + "context_window": 1000000, + "per_turn_token": 10361, + "response_id": "" + }, + "costs": [ + { + "model": "deepseek/deepseek-flash", + "cost": 0.00022199999999999998, + "timestamp": 1794386762.283774 + }, + { + "model": "deepseek/deepseek-flash", + "cost": 0.002211468, + "timestamp": 1794386762.5464773 + }, + { + "model": "deepseek/deepseek-flash", + "cost": 0.000151572, + "timestamp": 1794386790.3247352 + } + ], + "response_latencies": [ + { + "model": "deepseek/deepseek-flash", + "latency": 1.625413417816162, + "response_id": "5525774b-ccf4-4d3b-ac44-049a0f48894d" + }, + { + "model": "deepseek/deepseek-flash", + "latency": 1.8880395889282227, + "response_id": "fde20ca6-dfd3-4a62-8d4b-ea0d24d3b24f" + }, + { + "model": "deepseek/deepseek-flash", + "latency": 2.066305160522461, + "response_id": "b308785c-814f-49fb-8bd7-ee7da763497d" + } + ], + "token_usages": [ + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 140, + "completion_tokens": 150, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 140, + "context_window": 1000000, + "per_turn_token": 290, + "response_id": "5525774b-ccf4-4d3b-ac44-049a0f48894d" + }, + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 10125, + "completion_tokens": 127, + "cache_read_tokens": 3328, + "cache_write_tokens": 0, + "reasoning_tokens": 23, + "context_window": 1000000, + "per_turn_token": 10252, + "response_id": "fde20ca6-dfd3-4a62-8d4b-ea0d24d3b24f" + }, + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 10343, + "completion_tokens": 18, + "cache_read_tokens": 10112, + "cache_write_tokens": 0, + "reasoning_tokens": 0, + "context_window": 1000000, + "per_turn_token": 10361, + "response_id": "b308785c-814f-49fb-8bd7-ee7da763497d" + } + ] + }, + "condenser": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.0, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 0, + "completion_tokens": 0, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 0, + "context_window": 0, + "per_turn_token": 0, + "response_id": "" + }, + "costs": [], + "response_latencies": [], + "token_usages": [] + } + } + }, + "secret_registry": { + "secret_sources": {} + }, + "tags": {}, + "agent_state": {} + }, + "kind": "ConversationStateUpdateEvent" + } + } + }, + { + "direction": "ended", + "at": "2026-11-11T08:46:43Z", + "stopped": "duration", + "close": null + } + ] +} diff --git a/.pr/bugs/2026-11-11/session-socket-sync-first/pre6709-sync-first.json b/.pr/bugs/2026-11-11/session-socket-sync-first/pre6709-sync-first.json new file mode 100644 index 00000000000..774778d748a --- /dev/null +++ b/.pr/bugs/2026-11-11/session-socket-sync-first/pre6709-sync-first.json @@ -0,0 +1,704 @@ +{ + "summary": { + "opened": true, + "frames": 8, + "kinds": { + "sync": 1, + "transient:ConversationStateUpdateEvent": 2, + "durable:ObservationEvent": 1, + "error": 1, + "durable:ConversationStateUpdateEvent": 2, + "durable:MessageEvent": 1 + }, + "stopped": "duration", + "close": null + }, + "frames": [ + { + "direction": "opened", + "at": "2026-11-11T08:47:02Z" + }, + { + "direction": "sent", + "at": "2026-11-11T08:47:02Z", + "frame": { + "role": "assistant", + "content": [ + { + "type": "text", + "text": "QA_F07_RACE" + } + ] + } + }, + { + "direction": "received", + "at": "2026-11-11T08:47:28Z", + "kind": "sync", + "frame": { + "type": "sync", + "through_seq": 7 + } + }, + { + "direction": "received", + "at": "2026-11-11T08:47:28Z", + "kind": "transient:ConversationStateUpdateEvent", + "frame": { + "type": "transient", + "event": { + "id": "69743a9c-9823-4b8a-b0bf-c59986451d2c", + "timestamp": "2026-11-11T08:47:28.436166", + "source": "environment", + "key": "full_state", + "value": { + "id": "ed851f8e-e155-4514-a088-e2b21c107460", + "agent": { + "llm": { + "model": "deepseek/deepseek-flash", + "api_key": "**********", + "auth_type": "api_key", + "openrouter_site_url": "https://docs.all-hands.dev/", + "openrouter_app_name": "OpenHands", + "num_retries": 5, + "retry_multiplier": 8.0, + "retry_min_wait": 8, + "retry_max_wait": 64, + "timeout": 300, + "stream_idle_timeout": 300.0, + "max_message_chars": 30000, + "api_mode": "auto", + "capability_overrides": {}, + "stream": false, + "drop_params": true, + "disable_stop_word": false, + "caching_prompt": true, + "log_completions": false, + "log_completions_folder": "logs/completions", + "native_tool_calling": true, + "reasoning_effort": "high", + "enable_encrypted_reasoning": true, + "prompt_cache_retention": "24h", + "extended_thinking_budget": 200000, + "usage_id": "default", + "litellm_extra_body": {}, + "is_subscription": false + }, + "tools": [ + { + "name": "terminal", + "params": {} + }, + { + "name": "file_editor", + "params": {} + }, + { + "name": "task_tracker", + "params": {} + }, + { + "name": "browser_tool_set", + "params": {} + } + ], + "mcp_config": {}, + "include_default_tools": [ + "FinishTool", + "SwitchLLMTool" + ], + "agent_context": { + "skills": [], + "load_user_skills": false, + "load_public_skills": false, + "marketplace_path": "marketplaces/default.json", + "registered_marketplaces": [], + "load_project_skills": false, + "load_memory": false, + "disabled_skills": [], + "current_datetime": "2026-11-11T08:47:00.027158Z" + }, + "system_prompt_filename": "system_prompt.j2", + "security_policy_filename": "security_policy.j2", + "system_prompt_kwargs": { + "llm_security_analyzer": true + }, + "condenser": { + "llm": { + "model": "deepseek/deepseek-flash", + "api_key": "**********", + "auth_type": "api_key", + "openrouter_site_url": "https://docs.all-hands.dev/", + "openrouter_app_name": "OpenHands", + "num_retries": 5, + "retry_multiplier": 8.0, + "retry_min_wait": 8, + "retry_max_wait": 64, + "timeout": 300, + "stream_idle_timeout": 300.0, + "max_message_chars": 30000, + "api_mode": "auto", + "capability_overrides": {}, + "stream": false, + "drop_params": true, + "disable_stop_word": false, + "caching_prompt": true, + "log_completions": false, + "log_completions_folder": "logs/completions", + "native_tool_calling": true, + "reasoning_effort": "high", + "enable_encrypted_reasoning": true, + "prompt_cache_retention": "24h", + "extended_thinking_budget": 200000, + "usage_id": "condenser", + "litellm_extra_body": {}, + "is_subscription": false + }, + "max_size": 240, + "keep_first": 2, + "minimum_progress": 0.1, + "hard_context_reset_max_retries": 5, + "hard_context_reset_context_scaling": 0.8, + "kind": "LLMSummarizingCondenser" + }, + "tool_concurrency_limit": 1, + "kind": "Agent" + }, + "workspace": { + "working_dir": "/agent-server/20261111-084646-4c91/fixtures/workspaces/084700-013094", + "kind": "LocalWorkspace" + }, + "persistence_dir": "/agent-server/20261111-084646-4c91/server/workspace/conversations/ed851f8ee1554514a088e2b21c107460", + "max_iterations": 500, + "stuck_detection": true, + "execution_status": "running", + "confirmation_policy": { + "kind": "NeverConfirm" + }, + "activated_knowledge_skills": [], + "activated_path_rules": [], + "invoked_skills": [], + "blocked_actions": {}, + "blocked_messages": {}, + "last_user_message_id": "65bb433a-aa21-4465-9e2b-6cbb9f808e1c", + "leaf_event_id": "f282a7f7-5261-4686-9d85-6df60d4ab758", + "head_is_empty": false, + "stats": { + "usage_to_metrics": { + "default": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.0023857679999999995, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 10266, + "completion_tokens": 237, + "cache_read_tokens": 3328, + "cache_write_tokens": 0, + "reasoning_tokens": 123, + "context_window": 1000000, + "per_turn_token": 10291, + "response_id": "" + }, + "costs": [ + { + "model": "deepseek/deepseek-flash", + "cost": 0.00012839999999999998, + "timestamp": 1794386821.73052 + }, + { + "model": "deepseek/deepseek-flash", + "cost": 0.0022573679999999996, + "timestamp": 1794386822.5214221 + } + ], + "response_latencies": [ + { + "model": "deepseek/deepseek-flash", + "latency": 1.4140396118164062, + "response_id": "504d6c03-33db-4613-b489-31da3961e968" + }, + { + "model": "deepseek/deepseek-flash", + "latency": 2.2051618099212646, + "response_id": "d567d1e9-1353-44c5-b119-f3b863e9b596" + } + ], + "token_usages": [ + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 140, + "completion_tokens": 72, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 62, + "context_window": 1000000, + "per_turn_token": 212, + "response_id": "504d6c03-33db-4613-b489-31da3961e968" + }, + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 10126, + "completion_tokens": 165, + "cache_read_tokens": 3328, + "cache_write_tokens": 0, + "reasoning_tokens": 61, + "context_window": 1000000, + "per_turn_token": 10291, + "response_id": "d567d1e9-1353-44c5-b119-f3b863e9b596" + } + ] + }, + "condenser": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.0, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 0, + "completion_tokens": 0, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 0, + "context_window": 0, + "per_turn_token": 0, + "response_id": "" + }, + "costs": [], + "response_latencies": [], + "token_usages": [] + } + } + }, + "secret_registry": { + "secret_sources": {} + }, + "tags": {}, + "agent_state": {} + }, + "kind": "ConversationStateUpdateEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:47:28Z", + "kind": "durable:ObservationEvent", + "frame": { + "type": "durable", + "seq": 7, + "event": { + "id": "f282a7f7-5261-4686-9d85-6df60d4ab758", + "timestamp": "2026-11-11T08:47:28.430980", + "source": "environment", + "parent_id": "aae70fb2-20a4-47bd-a1c4-daca68a74a64", + "tool_name": "terminal", + "tool_call_id": "call_00_Tp7rNnwOMTW4Z7nbwlOV2932", + "observation": { + "content": [ + { + "cache_prompt": false, + "type": "text", + "text": "QA_DONE" + } + ], + "is_error": false, + "command": "sleep 25 && echo QA_DONE", + "exit_code": 0, + "timeout": false, + "metadata": { + "exit_code": 0, + "pid": -1, + "username": "root", + "hostname": "vm", + "working_dir": "/agent-server/20261111-084646-4c91/fixtures/workspaces/084700-013094", + "py_interpreter_path": "/usr/local/bin/python", + "prefix": "", + "suffix": "\n[The command completed with exit code 0.]" + }, + "full_output_save_dir": "/agent-server/20261111-084646-4c91/server/workspace/conversations/ed851f8ee1554514a088e2b21c107460/observations", + "kind": "TerminalObservation" + }, + "action_id": "aae70fb2-20a4-47bd-a1c4-daca68a74a64", + "extended_content": [], + "kind": "ObservationEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:47:28Z", + "kind": "error", + "frame": { + "type": "error", + "code": "AssertionError", + "detail": "Only user messages are allowed to be sent to the agent." + } + }, + { + "direction": "received", + "at": "2026-11-11T08:47:29Z", + "kind": "durable:ConversationStateUpdateEvent", + "frame": { + "type": "durable", + "seq": 8, + "event": { + "id": "5865b240-e90a-4cf8-9a11-89977caa5b17", + "timestamp": "2026-11-11T08:47:29.746109", + "source": "environment", + "parent_id": "f282a7f7-5261-4686-9d85-6df60d4ab758", + "key": "stats", + "value": { + "usage_to_metrics": { + "default": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.0025667399999999996, + "max_budget_per_task": null, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 20611, + "completion_tokens": 279, + "cache_read_tokens": 13440, + "cache_write_tokens": 0, + "reasoning_tokens": 147, + "context_window": 1000000, + "per_turn_token": 10387, + "response_id": "" + } + }, + "condenser": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.0, + "max_budget_per_task": null, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 0, + "completion_tokens": 0, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 0, + "context_window": 0, + "per_turn_token": 0, + "response_id": "" + } + } + } + }, + "kind": "ConversationStateUpdateEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:47:29Z", + "kind": "durable:MessageEvent", + "frame": { + "type": "durable", + "seq": 9, + "event": { + "id": "0a56ca1d-3f01-4292-9663-e7385366425b", + "timestamp": "2026-11-11T08:47:29.749438", + "source": "agent", + "parent_id": "f282a7f7-5261-4686-9d85-6df60d4ab758", + "llm_message": { + "role": "assistant", + "content": [ + { + "cache_prompt": false, + "type": "text", + "text": "Command executed successfully \u2014 output: `QA_DONE` (exit code 0)." + } + ], + "reasoning_content": "Command completed. Report briefly.\n\nThe user said run exactly this command and nothing else. I did. Then final message.", + "thinking_blocks": [] + }, + "llm_response_id": "72ba2403-f486-43f5-abf9-8e88622eba46", + "activated_skills": [], + "extended_content": [], + "kind": "MessageEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:47:29Z", + "kind": "durable:ConversationStateUpdateEvent", + "frame": { + "type": "durable", + "seq": 10, + "event": { + "id": "95c9daa4-ef62-40a3-99f6-2e3c3b47a46d", + "timestamp": "2026-11-11T08:47:29.752204", + "source": "environment", + "parent_id": "0a56ca1d-3f01-4292-9663-e7385366425b", + "key": "execution_status", + "value": "finished", + "kind": "ConversationStateUpdateEvent" + } + } + }, + { + "direction": "received", + "at": "2026-11-11T08:47:29Z", + "kind": "transient:ConversationStateUpdateEvent", + "frame": { + "type": "transient", + "event": { + "id": "5c7fb349-39dd-4710-a1e2-ff8add252e39", + "timestamp": "2026-11-11T08:47:29.759185", + "source": "environment", + "key": "full_state", + "value": { + "id": "ed851f8e-e155-4514-a088-e2b21c107460", + "agent": { + "llm": { + "model": "deepseek/deepseek-flash", + "api_key": "**********", + "auth_type": "api_key", + "openrouter_site_url": "https://docs.all-hands.dev/", + "openrouter_app_name": "OpenHands", + "num_retries": 5, + "retry_multiplier": 8.0, + "retry_min_wait": 8, + "retry_max_wait": 64, + "timeout": 300, + "stream_idle_timeout": 300.0, + "max_message_chars": 30000, + "api_mode": "auto", + "capability_overrides": {}, + "stream": false, + "drop_params": true, + "disable_stop_word": false, + "caching_prompt": true, + "log_completions": false, + "log_completions_folder": "logs/completions", + "native_tool_calling": true, + "reasoning_effort": "high", + "enable_encrypted_reasoning": true, + "prompt_cache_retention": "24h", + "extended_thinking_budget": 200000, + "usage_id": "default", + "litellm_extra_body": {}, + "is_subscription": false + }, + "tools": [ + { + "name": "terminal", + "params": {} + }, + { + "name": "file_editor", + "params": {} + }, + { + "name": "task_tracker", + "params": {} + }, + { + "name": "browser_tool_set", + "params": {} + } + ], + "mcp_config": {}, + "include_default_tools": [ + "FinishTool", + "SwitchLLMTool" + ], + "agent_context": { + "skills": [], + "load_user_skills": false, + "load_public_skills": false, + "marketplace_path": "marketplaces/default.json", + "registered_marketplaces": [], + "load_project_skills": false, + "load_memory": false, + "disabled_skills": [], + "current_datetime": "2026-11-11T08:47:00.027158Z" + }, + "system_prompt_filename": "system_prompt.j2", + "security_policy_filename": "security_policy.j2", + "system_prompt_kwargs": { + "llm_security_analyzer": true + }, + "condenser": { + "llm": { + "model": "deepseek/deepseek-flash", + "api_key": "**********", + "auth_type": "api_key", + "openrouter_site_url": "https://docs.all-hands.dev/", + "openrouter_app_name": "OpenHands", + "num_retries": 5, + "retry_multiplier": 8.0, + "retry_min_wait": 8, + "retry_max_wait": 64, + "timeout": 300, + "stream_idle_timeout": 300.0, + "max_message_chars": 30000, + "api_mode": "auto", + "capability_overrides": {}, + "stream": false, + "drop_params": true, + "disable_stop_word": false, + "caching_prompt": true, + "log_completions": false, + "log_completions_folder": "logs/completions", + "native_tool_calling": true, + "reasoning_effort": "high", + "enable_encrypted_reasoning": true, + "prompt_cache_retention": "24h", + "extended_thinking_budget": 200000, + "usage_id": "condenser", + "litellm_extra_body": {}, + "is_subscription": false + }, + "max_size": 240, + "keep_first": 2, + "minimum_progress": 0.1, + "hard_context_reset_max_retries": 5, + "hard_context_reset_context_scaling": 0.8, + "kind": "LLMSummarizingCondenser" + }, + "tool_concurrency_limit": 1, + "kind": "Agent" + }, + "workspace": { + "working_dir": "/agent-server/20261111-084646-4c91/fixtures/workspaces/084700-013094", + "kind": "LocalWorkspace" + }, + "persistence_dir": "/agent-server/20261111-084646-4c91/server/workspace/conversations/ed851f8ee1554514a088e2b21c107460", + "max_iterations": 500, + "stuck_detection": true, + "execution_status": "finished", + "confirmation_policy": { + "kind": "NeverConfirm" + }, + "activated_knowledge_skills": [], + "activated_path_rules": [], + "invoked_skills": [], + "blocked_actions": {}, + "blocked_messages": {}, + "last_user_message_id": "65bb433a-aa21-4465-9e2b-6cbb9f808e1c", + "leaf_event_id": "0a56ca1d-3f01-4292-9663-e7385366425b", + "head_is_empty": false, + "stats": { + "usage_to_metrics": { + "default": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.0025667399999999996, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 20611, + "completion_tokens": 279, + "cache_read_tokens": 13440, + "cache_write_tokens": 0, + "reasoning_tokens": 147, + "context_window": 1000000, + "per_turn_token": 10387, + "response_id": "" + }, + "costs": [ + { + "model": "deepseek/deepseek-flash", + "cost": 0.00012839999999999998, + "timestamp": 1794386821.73052 + }, + { + "model": "deepseek/deepseek-flash", + "cost": 0.0022573679999999996, + "timestamp": 1794386822.5214221 + }, + { + "model": "deepseek/deepseek-flash", + "cost": 0.000180972, + "timestamp": 1794386849.7457292 + } + ], + "response_latencies": [ + { + "model": "deepseek/deepseek-flash", + "latency": 1.4140396118164062, + "response_id": "504d6c03-33db-4613-b489-31da3961e968" + }, + { + "model": "deepseek/deepseek-flash", + "latency": 2.2051618099212646, + "response_id": "d567d1e9-1353-44c5-b119-f3b863e9b596" + }, + { + "model": "deepseek/deepseek-flash", + "latency": 1.2732162475585938, + "response_id": "72ba2403-f486-43f5-abf9-8e88622eba46" + } + ], + "token_usages": [ + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 140, + "completion_tokens": 72, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 62, + "context_window": 1000000, + "per_turn_token": 212, + "response_id": "504d6c03-33db-4613-b489-31da3961e968" + }, + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 10126, + "completion_tokens": 165, + "cache_read_tokens": 3328, + "cache_write_tokens": 0, + "reasoning_tokens": 61, + "context_window": 1000000, + "per_turn_token": 10291, + "response_id": "d567d1e9-1353-44c5-b119-f3b863e9b596" + }, + { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 10345, + "completion_tokens": 42, + "cache_read_tokens": 10112, + "cache_write_tokens": 0, + "reasoning_tokens": 24, + "context_window": 1000000, + "per_turn_token": 10387, + "response_id": "72ba2403-f486-43f5-abf9-8e88622eba46" + } + ] + }, + "condenser": { + "model_name": "deepseek/deepseek-flash", + "accumulated_cost": 0.0, + "accumulated_token_usage": { + "model": "deepseek/deepseek-flash", + "prompt_tokens": 0, + "completion_tokens": 0, + "cache_read_tokens": 0, + "cache_write_tokens": 0, + "reasoning_tokens": 0, + "context_window": 0, + "per_turn_token": 0, + "response_id": "" + }, + "costs": [], + "response_latencies": [], + "token_usages": [] + } + } + }, + "secret_registry": { + "secret_sources": {} + }, + "tags": {}, + "agent_state": {} + }, + "kind": "ConversationStateUpdateEvent" + } + } + }, + { + "direction": "ended", + "at": "2026-11-11T08:47:42Z", + "stopped": "duration", + "close": null + } + ] +} diff --git a/.pr/bugs/2026-11-11/session-socket-sync-first/sync_first.sh b/.pr/bugs/2026-11-11/session-socket-sync-first/sync_first.sh new file mode 100644 index 00000000000..849a0bdb32c --- /dev/null +++ b/.pr/bugs/2026-11-11/session-socket-sync-first/sync_first.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +# Usage: sync_first.sh TAG (AGENT_SERVER_VERIFY_RUN must be exported; deepseek preset applied) +# Starts a conversation whose agent runs `sleep 60`, waits until the sleep +# ActionEvent is persisted (run holds the state lock), then opens a session +# socket and sends (a) an invalid assistant message, (b) separately a valid user +# message, recording frame order. +set -u +TAG=$1 +C=control-agent-server +CID=$($C conversation start --prompt 'Use the terminal tool to run exactly this command and nothing else: sleep 60 . After it returns, reply DONE and finish.' --print-id) +echo "CID=$CID" +for i in $(seq 1 90); do + if $C conversation events "$CID" --kinds ActionEvent --contains 'sleep 60' --expect-min 1 >/dev/null 2>&1; then echo "sleep action seen after ${i}s"; break; fi + sleep 1 +done +$C api GET "/api/conversations/$CID" --field execution_status 2>/dev/null | tail -1 +# (a) invalid frame during the running step +$C ws listen "/sockets/session/$CID" --send '{"role":"assistant","content":[{"type":"text","text":"x"}]}' \ + --duration 6 --save "F07.sync-first/$TAG-invalid-$CID" --show 10 > "/repro/$TAG-invalid.out" +# (b) valid user message during the running step (run:false so it is just saved/queued) +$C ws listen "/sockets/session/$CID" --send '{"role":"user","content":[{"type":"text","text":"QA_VALID_DURING_STEP_'"$TAG"'"}]}' \ + --duration 4 --save "F07.sync-first/$TAG-valid-$CID" --show 10 > "/repro/$TAG-valid.out" +echo "CID=$CID" > "/repro/$TAG.cid" +# (c) invalid frame again, now listening until the sync frame finally arrives (after the sleep ends) +$C ws listen "/sockets/session/$CID" --send '{"role":"assistant","content":[{"type":"text","text":"x"}]}' \ + --until-kind sync --duration 120 --save "F07.sync-first/$TAG-invalid-until-sync-$CID" --show 10 > "/repro/$TAG-invalid-until-sync.out" +$C conversation wait "$CID" --until finished --timeout 240 > /dev/null +$C conversation events "$CID" --kinds MessageEvent --contains "QA_VALID_DURING_STEP_$TAG" --expect-min 1 --save "F07.sync-first/$TAG-valid-persisted" > "/repro/$TAG-valid-persisted.out" +echo "valid persisted exit=$?" diff --git a/.pr/bugs/2026-11-11/tmux-plain-follow-up/input1.txt b/.pr/bugs/2026-11-11/tmux-plain-follow-up/input1.txt new file mode 100644 index 00000000000..8a18a2397ea --- /dev/null +++ b/.pr/bugs/2026-11-11/tmux-plain-follow-up/input1.txt @@ -0,0 +1,25 @@ +mode: is_input=True terminal_type=None +===== OBS 1 cmd='python3' ===== +Python 3.11.18 (main, Oct 26 2026, 15:28:55) [GCC 13.3.0] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +[The command has no new output after 3 seconds. You may wait longer to see additional output by sending empty command '', send other commands to interact with the current process, send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command, or use the timeout parameter in terminal for future commands.] +===== OBS 2 cmd='print(6 * 7)' ===== +[Below is the output of the previous command.] +42 +>>> +[The command has no new output after 3 seconds. You may wait longer to see additional output by sending empty command '', send other commands to interact with the current process, send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command, or use the timeout parameter in terminal for future commands.] +===== OBS 3 cmd='exit()' ===== + +[The command completed with exit code 0.] +[Current working directory: /tmp/repl2ws-or9c4j2g] +[Python interpreter: /usr/local/bin/python] +[Command finished with exit code 0] +===== OBS 4 cmd='echo back-in-shell' ===== +back-in-shell +[The command completed with exit code 0.] +[Current working directory: /tmp/repl2ws-or9c4j2g] +[Python interpreter: /usr/local/bin/python] +[Command finished with exit code 0] +===== OBS 5 cmd=None ===== +done diff --git a/.pr/bugs/2026-11-11/tmux-plain-follow-up/plain1.txt b/.pr/bugs/2026-11-11/tmux-plain-follow-up/plain1.txt new file mode 100644 index 00000000000..8f1ad2afd30 --- /dev/null +++ b/.pr/bugs/2026-11-11/tmux-plain-follow-up/plain1.txt @@ -0,0 +1,51 @@ +mode: is_input=False terminal_type=None +===== OBS 1 cmd='python3' ===== +Python 3.11.18 (main, Oct 26 2026, 15:28:55) [GCC 13.3.0] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +[The command has no new output after 3 seconds. You may wait longer to see additional output by sending empty command '', send other commands to interact with the current process, send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command, or use the timeout parameter in terminal for future commands.] +===== OBS 2 cmd='print(6 * 7)' ===== +python3 +Python 3.11.18 (main, Oct 26 2026, 15:28:55) [GCC 13.3.0] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +KeyboardInterrupt +>>> clear +Traceback (most recent call last): + File "", line 1, in +NameError: name 'clear' is not defined +>>> print(6 * 7) +42 +>>> +[The command has no new output after 3 seconds. You may wait longer to see additional output by sending empty command '', send other commands to interact with the current process, send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command, or use the timeout parameter in terminal for future commands.] +===== OBS 3 cmd='exit()' ===== +python3 +Python 3.11.18 (main, Oct 26 2026, 15:28:55) [GCC 13.3.0] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +KeyboardInterrupt +>>> clear +Traceback (most recent call last): + File "", line 1, in +NameError: name 'clear' is not defined +>>> print(6 * 7) +42 +>>> +KeyboardInterrupt +>>> clear +Traceback (most recent call last): + File "", line 1, in +NameError: name 'clear' is not defined +>>> exit() +[The command completed with exit code 0.] +[Current working directory: /tmp/repl2ws-_vx6clbn] +[Python interpreter: /usr/local/bin/python] +[Command finished with exit code 0] +===== OBS 4 cmd='echo back-in-shell' ===== +back-in-shell +[The command completed with exit code 0.] +[Current working directory: /tmp/repl2ws-_vx6clbn] +[Python interpreter: /usr/local/bin/python] +[Command finished with exit code 0] +===== OBS 5 cmd=None ===== +done diff --git a/.pr/bugs/2026-11-11/tmux-plain-follow-up/repro.py b/.pr/bugs/2026-11-11/tmux-plain-follow-up/repro.py new file mode 100644 index 00000000000..82a8d404e49 --- /dev/null +++ b/.pr/bugs/2026-11-11/tmux-plain-follow-up/repro.py @@ -0,0 +1,34 @@ +import sys, tempfile +sys.path.insert(0, "/home/user/software-agent-sdk/.agents/skills/verify-sdk/programs") +from sdk_verify import make_llm +from openhands.sdk import Agent, Conversation, Tool +from openhands.sdk.event import ObservationEvent +from openhands.sdk.tool import register_tool +from openhands.tools.terminal import TerminalTool + +is_input = "--input" in sys.argv +ttype = None +for a in sys.argv: + if a.startswith("--type="): ttype = a.split("=",1)[1] +register_tool(TerminalTool.name, TerminalTool) +def step(cmd, inp=False): + args = {"command": cmd} + if inp: args["is_input"] = True + return {"tool": "terminal", "args": args} +script = [step("python3"), step("print(6 * 7)", is_input), step("exit()", is_input), + step("echo back-in-shell"), {"finish": "done"}] +params = {"no_change_timeout_seconds": 3} +if ttype: params["terminal_type"] = ttype +agent = Agent(llm=make_llm(script), tools=[Tool(name=TerminalTool.name, params=params)]) +ws = tempfile.mkdtemp(prefix="repl2ws-") +conv = Conversation(agent=agent, workspace=ws, visualizer=None) +print(f"mode: is_input={is_input} terminal_type={ttype}") +conv.send_message("go") +conv.run() +i = 0 +for ev in conv.state.events: + if isinstance(ev, ObservationEvent): + i += 1 + print(f"===== OBS {i} cmd={getattr(ev.observation,'command',None)!r} =====") + print("\n".join(getattr(c, "text", "") for c in ev.observation.to_llm_content)) +conv.close() diff --git a/.pr/bugs/2026-11-11/tmux-plain-follow-up/sdk-map-S04-interactive-terminal.log b/.pr/bugs/2026-11-11/tmux-plain-follow-up/sdk-map-S04-interactive-terminal.log new file mode 100644 index 00000000000..6a5143c242c --- /dev/null +++ b/.pr/bugs/2026-11-11/tmux-plain-follow-up/sdk-map-S04-interactive-terminal.log @@ -0,0 +1,86 @@ +$ export SDK_VERIFY_RUN=$(control-sdk new --field run) +control-sdk program S04/views --arg interactive \ + --check repl.status eq finished --check repl.waiting eq true --check repl.reply contains 42 \ + --check repl.collected_tool_messages eq 5 --check repl.collected_roles contains tool \ + --check prompt_is_input.waiting eq true --check prompt_is_input.reply contains "hello John 42" +{ + "ok": true, + "mode": "interactive", + "repl": { + "status": "finished", + "waiting": true, + "reply": "python3\nPython 3.11.18 (main, Oct 26 2026, 15:28:55) [GCC 13.3.0] on linux\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n>>>\nKeyboardInterrupt\n>>> clear\nTraceback (most recent", + "collected_roles": [ + "assistant", + "system", + "tool", + "user" + ], + "collected_tool_messages": 5 + }, + "prompt_is_input": { + "status": "finished", + "waiting": true, + "reply": "hello John 42\n[The command completed with exit code 0.]\n[Current working directory: /sdk/S04-it-1/work/S04-local-conversation-19", + "collected_roles": [ + "assistant", + "system", + "tool", + "user" + ], + "collected_tool_messages": 3 + }, + "prompt_plain": { + "status": "finished", + "waiting": true, + "reply": "bash: John: command not found\n[The command completed with exit code 127.]\n[Current working directory: /sdk/S04-it-1/work/S04-loc", + "collected_roles": [ + "assistant", + "system", + "tool", + "user" + ], + "collected_tool_messages": 3 + }, + "_exit": 0, + "_seconds": 33.6, + "checks": [ + { + "check": "repl.status eq finished", + "ok": true, + "actual": "finished" + }, + { + "check": "repl.waiting eq true", + "ok": true, + "actual": "true" + }, + { + "check": "repl.reply contains 42", + "ok": false, + "actual": "python3\nPython 3.11.18 (main, Oct 26 2026, 15:28:55) [GCC 13.3.0] on linux\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n>>>\nKeyboardInterrupt\n>>> clear\nTraceback (most recent" + }, + { + "check": "repl.collected_tool_messages eq 5", + "ok": true, + "actual": "5" + }, + { + "check": "repl.collected_roles contains tool", + "ok": true, + "actual": "[\"assistant\", \"system\", \"tool\", \"user\"]" + }, + { + "check": "prompt_is_input.waiting eq true", + "ok": true, + "actual": "true" + }, + { + "check": "prompt_is_input.reply contains hello John 42", + "ok": true, + "actual": "hello John 42\n[The command completed with exit code 0.]\n[Current working directory: /sdk/S04-it-1/work/S04-local-conversation-19" + } + ] +} + +[exit 1 after 34s] diff --git a/.pr/bugs/2026-11-11/tmux-plain-follow-up/subproc_plain.txt b/.pr/bugs/2026-11-11/tmux-plain-follow-up/subproc_plain.txt new file mode 100644 index 00000000000..83cb5614e9c --- /dev/null +++ b/.pr/bugs/2026-11-11/tmux-plain-follow-up/subproc_plain.txt @@ -0,0 +1,26 @@ +mode: is_input=False terminal_type=subprocess +===== OBS 1 cmd='python3' ===== +[?2004lPython 3.11.18 (main, Oct 26 2026, 15:28:55) [GCC 13.3.0] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +[The command has no new output after 3 seconds. You may wait longer to see additional output by sending empty command '', send other commands to interact with the current process, send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command, or use the timeout parameter in terminal for future commands.] +===== OBS 2 cmd='print(6 * 7)' ===== +[An error occurred during execution.] + +[Below is the output of the previous command.] + +[Your command "print(6 * 7)" is NOT executed. The previous command is still running - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see additional output by sending empty command '', send other commands to interact with the current process, send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command, or use the timeout parameter in terminal for future commands.] +===== OBS 3 cmd='exit()' ===== +[An error occurred during execution.] + +[Below is the output of the previous command.] + +[Your command "exit()" is NOT executed. The previous command is still running - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see additional output by sending empty command '', send other commands to interact with the current process, send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command, or use the timeout parameter in terminal for future commands.] +===== OBS 4 cmd='echo back-in-shell' ===== +[An error occurred during execution.] + +[Below is the output of the previous command.] + +[Your command "echo back-in-shell" is NOT executed. The previous command is still running - You CANNOT send new commands until the previous command is completed. By setting `is_input` to `true`, you can interact with the current process: You may wait longer to see additional output by sending empty command '', send other commands to interact with the current process, send keys ("C-c", "C-z", "C-d") to interrupt/kill the previous command before sending your new command, or use the timeout parameter in terminal for future commands.] +===== OBS 5 cmd=None ===== +done diff --git a/.pr/bugs/2026-11-11/ts-acp-profile-effort/acp-profile-response.json b/.pr/bugs/2026-11-11/ts-acp-profile-effort/acp-profile-response.json new file mode 100644 index 00000000000..cb5f4acd6e3 --- /dev/null +++ b/.pr/bugs/2026-11-11/ts-acp-profile-effort/acp-profile-response.json @@ -0,0 +1 @@ +{"name":"qa-f21-acp","profile":{"schema_version":4,"agent_kind":"acp","id":"6aab8348-0669-4e8d-b483-cead04f86e17","name":"qa-f21-acp","revision":0,"mcp_server_refs":null,"secret_refs":["OPENAI_API_KEY"],"integration_refs":null,"acp_server":"codex","acp_model":"gpt-5.5","acp_effort":null,"acp_session_mode":null,"acp_prompt_timeout":1800.0,"acp_startup_timeout":90.0,"acp_command":null,"acp_args":null}} \ No newline at end of file diff --git a/.pr/bugs/2026-11-11/ts-acp-profile-effort/f21-contract-tsc.txt b/.pr/bugs/2026-11-11/ts-acp-profile-effort/f21-contract-tsc.txt new file mode 100644 index 00000000000..23b02d2d766 --- /dev/null +++ b/.pr/bugs/2026-11-11/ts-acp-profile-effort/f21-contract-tsc.txt @@ -0,0 +1 @@ +../../../agent-server/20261111-090144-e780-map-f21/fixtures/qa-f21-types/contract.ts(5,224): error TS2353: Object literal may only specify known properties, and '"acp_effort"' does not exist in type 'Shape'. diff --git a/.pr/bugs/2026-11-11/ts-acp-profile-effort/f21-exported-tsc.txt b/.pr/bugs/2026-11-11/ts-acp-profile-effort/f21-exported-tsc.txt new file mode 100644 index 00000000000..96d898cb91d --- /dev/null +++ b/.pr/bugs/2026-11-11/ts-acp-profile-effort/f21-exported-tsc.txt @@ -0,0 +1 @@ +../../../agent-server/20261111-090144-e780-map-f21/fixtures/qa-f21-types/exported.ts(5,224): error TS2353: Object literal may only specify known properties, and '"acp_effort"' does not exist in type 'Shape'. diff --git a/.pr/map-maintenance/2026-11-11/agent-server-report.md b/.pr/map-maintenance/2026-11-11/agent-server-report.md new file mode 100644 index 00000000000..5afc9779be4 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/agent-server-report.md @@ -0,0 +1,1120 @@ +| Feature | Run | Checkout | Result | Entry point | Expected | Actual | Evidence | +|---|---|---|---|---|---|---|---| +| F07.reject-non-user | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | fail | map run F07-session-socket.md: Non-user role (`F07.reject-non-user`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-08-non-user-role-f07-reject-non-user.json | +| F21.ts-payload-types | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | fail | map run F21-agent-profiles.md: TypeScript payload types (`F21.ts-payload-types`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 2 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-31-typescript-payload-types-f21-ts-payload-.json | +| F24.offline-safe | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | fail | map run F24-openai-subscription.md: Offline-safe routes (`F24.offline-safe`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 3 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-16-offline-safe-routes-f24-offline-safe.json | +| F27.custom-tool-missing-on-resume | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | fail | map run F27-hooks-subagents-tools.md: Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-25-tool-module-gone-on-resume-f27-custom-to.json | +| F27.import-modules | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | fail | map run F27-hooks-subagents-tools.md: Startup imports (`F27.import-modules`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-24-startup-imports-f27-import-modules.json | +| F33.conversation-confirmation | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | fail | map run F33-sdk-remote-client.md: Confirm or reject (`F33.conversation-confirmation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-27-confirm-or-reject-f33-conversation-confi.json | +| F33.conversation-pause | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | fail | map run F33-sdk-remote-client.md: Pause a blocking run (`F33.conversation-pause`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-28-pause-a-blocking-run-f33-conversation-pa.json | +| F33.conversation-reconnect | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | fail | map run F33-sdk-remote-client.md: A live conversation across a restart (`F33.conversation-reconnect`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-36-a-live-conversation-across-a-restart-f33.json | +| F33.conversation-restart | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | fail | map run F33-sdk-remote-client.md: Reattach after a restart and continue (`F33.conversation-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-38-reattach-after-a-restart-and-continue-f3.json | +| F33.conversation-run-timeout | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | fail | map run F33-sdk-remote-client.md: Time out, then wait again (`F33.conversation-run-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-30-time-out-then-wait-again-f33-conversatio.json | +| F33.conversation-secrets | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | fail | map run F33-sdk-remote-client.md: Secrets for the agent (`F33.conversation-secrets`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-33-secrets-for-the-agent-f33-conversation-s.json | +| F33.custom-tool-module | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | fail | map run F33-sdk-remote-client.md: A custom tool module (`F33.custom-tool-module`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-39-a-custom-tool-module-f33-custom-tool-mod.json | +| F33.ws-unavailable | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | fail | map run F33-sdk-remote-client.md: No events socket (`F33.ws-unavailable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-35-no-events-socket-f33-ws-unavailable.json | +| F09.acp-effort-codex | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | blocked | map run F09-model-switching-plugins.md: Codex effort levels (`F09.acp-effort-codex`), blocked: needs a ChatGPT subscription login for codex-acp (`CODEX_AUTH_JSON`). | prerequisite available | blocked (see the bullet) | | +| F09.acp-effort-codex | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | blocked | map run F09-model-switching-plugins.md: Codex effort levels (`F09.acp-effort-codex`), blocked: needs a ChatGPT subscription login for codex-acp (`CODEX_AUTH_JSON`). | prerequisite available | blocked (see the bullet) | | +| F12.docker-runtime | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | blocked | map run F12-runtime-and-credentials.md: Docker runtime with a daemon (`F12.docker-runtime`). | prerequisite available | blocked: needs docker | | +| F12.docker-startup-timeout | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | blocked | map run F12-runtime-and-credentials.md: Docker runtime startup deadline (`F12.docker-startup-timeout`). | prerequisite available | blocked: needs docker | | +| F15.pr-status-github | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | blocked | map run F15-git.md: Live pull-request status (`F15.pr-status-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`. | prerequisite available | blocked (see the bullet) | | +| F15.pr-status-github | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | blocked | map run F15-git.md: Live pull-request status (`F15.pr-status-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`. | prerequisite available | blocked (see the bullet) | | +| F15.pr-status-github | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | blocked | map run F15-git.md: Live pull-request status (`F15.pr-status-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`. | prerequisite available | blocked (see the bullet) | | +| F15.repo-search-github | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | blocked | map run F15-git.md: Live GitHub listing (`F15.repo-search-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`. | prerequisite available | blocked (see the bullet) | | +| F15.repo-search-github | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | blocked | map run F15-git.md: Live GitHub listing (`F15.repo-search-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`. | prerequisite available | blocked (see the bullet) | | +| F15.repo-search-github | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | blocked | map run F15-git.md: Live GitHub listing (`F15.repo-search-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`. | prerequisite available | blocked (see the bullet) | | +| F16.vscode-folder-encoding | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | blocked | map run F16-workspace-serving-and-vscode.md: Folder with `&` and `#` (`F16.vscode-folder-encoding`). | prerequisite available | blocked: needs vscode | | +| F16.vscode-running | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | blocked | map run F16-workspace-serving-and-vscode.md: VS Code actually running (`F16.vscode-running`). | prerequisite available | blocked: needs vscode | | +| F16.vscode-url-shape | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | blocked | map run F16-workspace-serving-and-vscode.md: URL shape (`F16.vscode-url-shape`). | prerequisite available | blocked: needs vscode | | +| F24.login-complete | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | blocked | map run F24-openai-subscription.md: Complete a login (`F24.login-complete`), blocked. | prerequisite available | blocked (see the bullet) | | +| F27.browser-playwright-path | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | blocked | map run F27-hooks-subagents-tools.md: Playwright browsers directory (`F27.browser-playwright-path`), blocked on a machine with Chromium at a standard path (needs Chromium only in a Playwright directory). | prerequisite available | blocked (see the bullet) | | +| F27.browser-probe | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | blocked | map run F27-hooks-subagents-tools.md: Browser probe (`F27.browser-probe`), blocked on a machine with Chromium at a standard path (needs Chromium only in a Playwright directory). | prerequisite available | blocked (see the bullet) | | +| F31.storage-retention | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | blocked | map run F31-persistence-and-limits.md: Storage retention and disk budget (`F31.storage-retention`). | prerequisite available | blocked: needs docker | | +| F33.workspace-runtime-lifecycle | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | blocked | map run F33-sdk-remote-client.md: Runtime key and release (`F33.workspace-runtime-lifecycle`). | prerequisite available | blocked: needs docker | | +| F03.init-delivered-cors-origins | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | xfail | map run F03-deferred-init.md: Delivered CORS origins (`F03.init-delivered-cors-origins`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-18-delivered-cors-origins-f03-init-delivere.json | +| F04.create-initial-message-run-false | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F04-conversation-lifecycle.md: `initial_message.run: false` (`F04.create-initial-message-run-false`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-07-initial-message-run-false-f04-create-ini.json | +| F04.create-secrets-get | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F04-conversation-lifecycle.md: Start-time secret visible to GET (`F04.create-secrets-get`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-13-start-time-secret-visible-to-get-f04-cre.json | +| F05.pause-running-crash | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F05-run-pause-interrupt.md: Crash after a mid-call pause (`F05.pause-running-crash`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-31-crash-after-a-mid-call-pause-f05-pause-r.json | +| F05.pause-running-get | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F05-run-pause-interrupt.md: GET after a mid-call pause (`F05.pause-running-get`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-16-get-after-a-mid-call-pause-f05-pause-run.json | +| F06.send-non-user-role | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F06-conversation-events.md: Non-user role (`F06.send-non-user-role`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-14-non-user-role-f06-send-non-user-role-kno.json | +| F06.ws-delete-close | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F06-conversation-events.md: Deleting a conversation closes its sockets (`F06.ws-delete-close`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-17-deleting-a-conversation-closes-its-socke.json | +| F07.deleted-close | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F07-session-socket.md: Deleting closes the socket (`F07.deleted-close`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-24-deleting-closes-the-socket-f07-deleted-c.json | +| F07.sync-first-during-run | 20261111-091133-869f-map-f07 | edff6711505acb0f88d90dbbaeaec654318e4289 | xfail | map run F07-session-socket.md: Sync first while a run holds the lock (`F07.sync-first-during-run`), known bug #6745. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-091133-869f-map-f07/evidence/map-run/F07-session-socket-01-sync-first-while-a-run-holds-the-lock-f0.json | +| F07.sync-first-during-run | 20261111-091219-fa68-map-f07 | edff6711505acb0f88d90dbbaeaec654318e4289 | xfail | map run F07-session-socket.md: Sync first while a run holds the lock (`F07.sync-first-during-run`), known bug #6745. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-091219-fa68-map-f07/evidence/map-run/F07-session-socket-01-sync-first-while-a-run-holds-the-lock-f0.json | +| F08.secrets-unsaved-restart | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F08-confirmation-security-secrets.md: Unsaved secret restart (`F08.secrets-unsaved-restart`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-23-unsaved-secret-restart-f08-secrets-unsav.json | +| F08.secrets-visible | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F08-confirmation-security-secrets.md: Runtime secret visible to GET (`F08.secrets-visible`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-13-runtime-secret-visible-to-get-f08-secret.json | +| F09.acp-model-live-unknown | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F09-model-switching-plugins.md: Unknown model on a live session (`F09.acp-model-live-unknown`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-19-unknown-model-on-a-live-session-f09-acp-.json | +| F09.acp-model-live-unknown | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | xfail | map run F09-model-switching-plugins.md: Unknown model on a live session (`F09.acp-model-live-unknown`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-19-unknown-model-on-a-live-session-f09-acp-.json | +| F09.acp-model-live-unknown | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F09-model-switching-plugins.md: Unknown model on a live session (`F09.acp-model-live-unknown`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-19-unknown-model-on-a-live-session-f09-acp-.json | +| F09.load-plugin-twice | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F09-model-switching-plugins.md: Load the same plugin twice (`F09.load-plugin-twice`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-26-load-the-same-plugin-twice-f09-load-plug.json | +| F09.load-plugin-twice | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | xfail | map run F09-model-switching-plugins.md: Load the same plugin twice (`F09.load-plugin-twice`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-35-load-the-same-plugin-twice-f09-load-plug.json | +| F09.load-plugin-twice | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F09-model-switching-plugins.md: Load the same plugin twice (`F09.load-plugin-twice`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-35-load-the-same-plugin-twice-f09-load-plug.json | +| F09.switch-llm-before-first-run | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F09-model-switching-plugins.md: Switch before the first run (`F09.switch-llm-before-first-run`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-11-switch-before-the-first-run-f09-switch-l.json | +| F09.switch-llm-before-first-run | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | xfail | map run F09-model-switching-plugins.md: Switch before the first run (`F09.switch-llm-before-first-run`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-11-switch-before-the-first-run-f09-switch-l.json | +| F09.switch-llm-before-first-run | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F09-model-switching-plugins.md: Switch before the first run (`F09.switch-llm-before-first-run`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-11-switch-before-the-first-run-f09-switch-l.json | +| F09.switch-llm-mid-run-get | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F09-model-switching-plugins.md: GET right after a mid-run switch (`F09.switch-llm-mid-run-get`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-13-get-right-after-a-mid-run-switch-f09-swi.json | +| F09.switch-llm-mid-run-get | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | xfail | map run F09-model-switching-plugins.md: GET right after a mid-run switch (`F09.switch-llm-mid-run-get`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-13-get-right-after-a-mid-run-switch-f09-swi.json | +| F09.switch-llm-mid-run-get | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F09-model-switching-plugins.md: GET right after a mid-run switch (`F09.switch-llm-mid-run-get`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-13-get-right-after-a-mid-run-switch-f09-swi.json | +| F09.switch-profile-edited | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F09-model-switching-plugins.md: Switch to an edited profile (`F09.switch-profile-edited`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-09-switch-to-an-edited-profile-f09-switch-p.json | +| F09.switch-profile-edited | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | xfail | map run F09-model-switching-plugins.md: Switch to an edited profile (`F09.switch-profile-edited`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-09-switch-to-an-edited-profile-f09-switch-p.json | +| F09.switch-profile-edited | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F09-model-switching-plugins.md: Switch to an edited profile (`F09.switch-profile-edited`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-09-switch-to-an-edited-profile-f09-switch-p.json | +| F10.fork-keeps-credentials | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F10-fork-navigate-condense-ask.md: The fork keeps the model keys (`F10.fork-keeps-credentials`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-08-the-fork-keeps-the-model-keys-f10-fork-k.json | +| F10.fork-run-memory | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | xfail | map run F10-fork-navigate-condense-ask.md: A fork runs with the source's memory (`F10.fork-run-memory`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-09-a-fork-runs-with-the-source-s-memory-f10.json | +| F12.sandbox-pause-websocket | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | xfail | map run F12-runtime-and-credentials.md: Open sockets across a pause (`F12.sandbox-pause-websocket`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-10-open-sockets-across-a-pause-f12-sandbox-.json | +| F13.scoped-idle-eviction | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | xfail | map run F13-bash.md: Idle eviction kills a running command (`F13.scoped-idle-eviction`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-29-idle-eviction-kills-a-running-command-f1.json | +| F14.upload-atomic | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | xfail | map run F14-files.md: Failed overwrite (`F14.upload-atomic`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-06-failed-overwrite-f14-upload-atomic-known.json | +| F16.docker-root-cookie | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | xfail | map run F16-workspace-serving-and-vscode.md: Docker-mode root and the cookie (`F16.docker-root-cookie`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-25-docker-mode-root-and-the-cookie-f16-dock.json | +| F19.test-remote-bearer | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | xfail | map run F19-mcp.md: Remote server behind a bearer token (`F19.test-remote-bearer`), known bug #6687. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-13-remote-server-behind-a-bearer-token-f19-.json | +| F20.validate-bad-key-type | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | xfail | map run F20-llm-profiles.md: DeepSeek 401 type (`F20.validate-bad-key-type`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-15-deepseek-401-type-f20-validate-bad-key-t.json | +| F26.conversation-resume-disabled | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | xfail | map run F26-plugins.md: Resume after disabling (`F26.conversation-resume-disabled`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-21-resume-after-disabling-f26-conversation-.json | +| F27.custom-tool-missing-on-resume | 20261111-084831-0bb5-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F27-hooks-subagents-tools.md: Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084831-0bb5-map-f27/evidence/map-run/F27-hooks-subagents-tools-02-tool-module-gone-on-resume-f27-custom-to.json | +| F27.custom-tool-missing-on-resume | 20261111-090008-47b9-map-f27 | 391b5546481d511261a4850e458b487478c663f0 | xfail | map run F27-hooks-subagents-tools.md: Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090008-47b9-map-f27/evidence/map-run/F27-hooks-subagents-tools-02-tool-module-gone-on-resume-f27-custom-to.json | +| F27.custom-tool-missing-on-resume | 20261111-090141-d9c4-map-f27 | 391b5546481d511261a4850e458b487478c663f0 | xfail | map run F27-hooks-subagents-tools.md: Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-090141-d9c4-map-f27/evidence/map-run/F27-hooks-subagents-tools-02-tool-module-gone-on-resume-f27-custom-to.json | +| F30.chat-reuse-concurrent | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F30-openai-gateway.md: Overlapping turns on one conversation (`F30.chat-reuse-concurrent`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-15-overlapping-turns-on-one-conversation-f3.json | +| F31.takeover-keeps-metadata | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F31-persistence-and-limits.md: Takeover keeps newer metadata (`F31.takeover-keeps-metadata`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-11-takeover-keeps-newer-metadata-f31-takeov.json | +| F32.deferred-init-flag | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F32-webhooks-and-telemetry.md: Deferred pods are labelled as such (`F32.deferred-init-flag`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-29-deferred-pods-are-labelled-as-such-f32-d.json | +| F32.slow-receiver | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F32-webhooks-and-telemetry.md: A slow receiver blocks the next run (`F32.slow-receiver`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-15-a-slow-receiver-blocks-the-next-run-f32-.json | +| F34.workspace-binary-download | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | xfail | map run F34-typescript-client.md: Binary download through RemoteWorkspace (`F34.workspace-binary-download`), known bug. | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 1 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-05-binary-download-through-remoteworkspace-.json | +| F01.alive | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: Liveness (`F01.alive`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-01-liveness-f01-alive.json | +| F01.debug-tracebacks | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: DEBUG tracebacks (`F01.debug-tracebacks`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-10-debug-tracebacks-f01-debug-tracebacks.json | +| F01.health | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: Health (`F01.health`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-02-health-f01-health.json | +| F01.idle-time | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: Idle time (`F01.idle-time`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-07-idle-time-f01-idle-time.json | +| F01.openapi | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: OpenAPI document (`F01.openapi`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-08-openapi-document-f01-openapi.json | +| F01.preload-tools | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: Tool preload (`F01.preload-tools`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-11-tool-preload-f01-preload-tools.json | +| F01.ready | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: Readiness (`F01.ready`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-03-readiness-f01-ready.json | +| F01.root-info | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: Root alias (`F01.root-info`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-05-root-alias-f01-root-info.json | +| F01.server-info | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: Server info (`F01.server-info`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-04-server-info-f01-server-info.json | +| F01.static-files | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: Static files (`F01.static-files`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-09-static-files-f01-static-files.json | +| F01.uptime-restart | 20261111-085419-c513-map-f01 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F01-server-status.md: Uptime across a restart (`F01.uptime-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085419-c513-map-f01/evidence/map-run/F01-server-status-06-uptime-across-a-restart-f01-uptime-resta.json | +| F02.cors-configured-origins | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Configured CORS origins (`F02.cors-configured-origins`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-15-configured-cors-origins-f02-cors-configu.json | +| F02.cors-default-origins | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Default CORS origins (`F02.cors-default-origins`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-14-default-cors-origins-f02-cors-default-or.json | +| F02.cors-workspace-routes | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Workspace routes accept any origin (`F02.cors-workspace-routes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-16-workspace-routes-accept-any-origin-f02-c.json | +| F02.default-bind | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Default bind address (`F02.default-bind`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-19-default-bind-address-f02-default-bind.json | +| F02.http-header-only | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Header only (`F02.http-header-only`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-02-header-only-f02-http-header-only.json | +| F02.http-key-required | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Key required on HTTP (`F02.http-key-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-01-key-required-on-http-f02-http-key-requir.json | +| F02.keys-hidden-from-commands | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Keys stay out of commands (`F02.keys-hidden-from-commands`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-13-keys-stay-out-of-commands-f02-keys-hidde.json | +| F02.legacy-env-key | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Legacy `SESSION_API_KEY` (`F02.legacy-env-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-18-legacy-session-api-key-f02-legacy-env-ke.json | +| F02.multiple-keys | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: A second key (`F02.multiple-keys`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-11-a-second-key-f02-multiple-keys.json | +| F02.no-auth-mode | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Unauthenticated mode (`F02.no-auth-mode`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-17-unauthenticated-mode-f02-no-auth-mode.json | +| F02.rotate-key | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Rotate the key (`F02.rotate-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-12-rotate-the-key-f02-rotate-key.json | +| F02.workspace-cookie-access | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: The cookie alone opens workspace files (`F02.workspace-cookie-access`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-09-the-cookie-alone-opens-workspace-files-f.json | +| F02.workspace-cookie-root-path | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Cookie path follows the prefix (`F02.workspace-cookie-root-path`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-20-cookie-path-follows-the-prefix-f02-works.json | +| F02.workspace-session-delete | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Delete the workspace cookie (`F02.workspace-session-delete`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-10-delete-the-workspace-cookie-f02-workspac.json | +| F02.workspace-session-ipv6-loopback | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: IPv6 loopback is a secure context (`F02.workspace-session-ipv6-loopback`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-21-ipv6-loopback-is-a-secure-context-f02-wo.json | +| F02.workspace-session-mint | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Mint the workspace cookie (`F02.workspace-session-mint`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-08-mint-the-workspace-cookie-f02-workspace-.json | +| F02.ws-auth-timeout | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Auth timeout (`F02.ws-auth-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-05-auth-timeout-f02-ws-auth-timeout.json | +| F02.ws-first-frame | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: First-frame auth on every socket (`F02.ws-first-frame`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-03-first-frame-auth-on-every-socket-f02-ws-.json | +| F02.ws-first-frame-reject | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Rejected first frames (`F02.ws-first-frame-reject`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-04-rejected-first-frames-f02-ws-first-frame.json | +| F02.ws-legacy-key | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Deprecated query and header keys (`F02.ws-legacy-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-06-deprecated-query-and-header-keys-f02-ws-.json | +| F02.ws-legacy-reject | 20261111-085504-7764-map-f02 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F02-auth-and-sessions.md: Wrong legacy key fails the handshake (`F02.ws-legacy-reject`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085504-7764-map-f02/evidence/map-run/F02-auth-and-sessions-07-wrong-legacy-key-fails-the-handshake-f02.json | +| F03.dormant-boot | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Dormant boot (`F03.dormant-boot`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-01-dormant-boot-f03-dormant-boot.json | +| F03.dormant-gates-api | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Dormant gate (`F03.dormant-gates-api`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-02-dormant-gate-f03-dormant-gates-api.json | +| F03.dormant-gates-sockets | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Dormant sockets (`F03.dormant-gates-sockets`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-03-dormant-sockets-f03-dormant-gates-socket.json | +| F03.dormant-gates-v1 | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Dormant /v1 and app backends (`F03.dormant-gates-v1`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-04-dormant-v1-and-app-backends-f03-dormant-.json | +| F03.gate-before-auth | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Readiness precedes auth (`F03.gate-before-auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-23-readiness-precedes-auth-f03-gate-before-.json | +| F03.init-applies-env | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Init env reaches bash (`F03.init-applies-env`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-12-init-env-reaches-bash-f03-init-applies-e.json | +| F03.init-child-launcher | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Child launcher after init (`F03.init-child-launcher`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-21-child-launcher-after-init-f03-init-child.json | +| F03.init-delivered-max-runs | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Delivered run capacity (`F03.init-delivered-max-runs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-17-delivered-run-capacity-f03-init-delivere.json | +| F03.init-delivered-webhooks | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Delivered webhooks (`F03.init-delivered-webhooks`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-10-delivered-webhooks-f03-init-delivered-we.json | +| F03.init-delivered-worktree-root | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Delivered worktree root (`F03.init-delivered-worktree-root`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-16-delivered-worktree-root-f03-init-deliver.json | +| F03.init-enforces-session-keys | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Delivered keys are enforced (`F03.init-enforces-session-keys`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-11-delivered-keys-are-enforced-f03-init-enf.json | +| F03.init-failure-rollback | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Failure rollback (`F03.init-failure-rollback`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-08-failure-rollback-f03-init-failure-rollba.json | +| F03.init-key-auth | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Init key auth (`F03.init-key-auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-05-init-key-auth-f03-init-key-auth.json | +| F03.init-keys-gate-bash-socket | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Delivered keys gate the bash socket (`F03.init-keys-gate-bash-socket`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-13-delivered-keys-gate-the-bash-socket-f03-.json | +| F03.init-open-without-secret | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Open init without a secret (`F03.init-open-without-secret`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-20-open-init-without-a-secret-f03-init-open.json | +| F03.init-opens-events-socket-and-v1 | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Events socket and /v1 open after init (`F03.init-opens-events-socket-and-v1`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-14-events-socket-and-v1-open-after-init-f03.json | +| F03.init-redacts-secret-input | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Input redaction (`F03.init-redacts-secret-input`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-07-input-redaction-f03-init-redacts-secret-.json | +| F03.init-second-call-400 | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Second init is 400 (`F03.init-second-call-400`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-19-second-init-is-400-f03-init-second-call-.json | +| F03.init-state-not-persisted | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Init state is not persisted (`F03.init-state-not-persisted`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-22-init-state-is-not-persisted-f03-init-sta.json | +| F03.init-success | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Successful init (`F03.init-success`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-09-successful-init-f03-init-success.json | +| F03.init-validation | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Init validation (`F03.init-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-06-init-validation-f03-init-validation.json | +| F03.init-web-url-root-path | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Delivered web URL sets the root path (`F03.init-web-url-root-path`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-15-delivered-web-url-sets-the-root-path-f03.json | +| F03.not-deferred-404 | 20261111-085450-0188-map-f03 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F03-deferred-init.md: Not deferred is 404 (`F03.not-deferred-404`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085450-0188-map-f03/evidence/map-run/F03-deferred-init-24-not-deferred-is-404-f03-not-deferred-404.json | +| F04.batch-get | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Batch get (`F04.batch-get`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-19-batch-get-f04-batch-get.json | +| F04.batch-get-limit | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: 100 ids in one batch (`F04.batch-get-limit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-20-100-ids-in-one-batch-f04-batch-get-limit.json | +| F04.child-capacity | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Child launch at capacity (`F04.child-capacity`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-36-child-launch-at-capacity-f04-child-capac.json | +| F04.child-errors | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Child launch errors (`F04.child-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-35-child-launch-errors-f04-child-errors.json | +| F04.child-start | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Start a child (`F04.child-start`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-33-start-a-child-f04-child-start.json | +| F04.child-unborn | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Default child in a fresh workspace (`F04.child-unborn`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-34-default-child-in-a-fresh-workspace-f04-c.json | +| F04.count | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Count (`F04.count`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-23-count-f04-count.json | +| F04.create-agent-profile | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Stored Agent Profile (`F04.create-agent-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-03-stored-agent-profile-f04-create-agent-pr.json | +| F04.create-idempotent | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Idempotent create with a client id (`F04.create-idempotent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-06-idempotent-create-with-a-client-id-f04-c.json | +| F04.create-inline-agent | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Inline agent (`F04.create-inline-agent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-02-inline-agent-f04-create-inline-agent.json | +| F04.create-observability | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Observability headers (`F04.create-observability`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-10-observability-headers-f04-create-observa.json | +| F04.create-observability-precedence | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Body versus headers (`F04.create-observability-precedence`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-11-body-versus-headers-f04-create-observabi.json | +| F04.create-parent | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Parent and child (`F04.create-parent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-08-parent-and-child-f04-create-parent.json | +| F04.create-secrets | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Start-time secrets (`F04.create-secrets`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-12-start-time-secrets-f04-create-secrets.json | +| F04.create-secrets-plain | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Plain-string secret value (`F04.create-secrets-plain`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-14-plain-string-secret-value-f04-create-sec.json | +| F04.create-settings | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Create from the saved settings (`F04.create-settings`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-01-create-from-the-saved-settings-f04-creat.json | +| F04.create-validation | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Create validation (`F04.create-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-04-create-validation-f04-create-validation.json | +| F04.create-validation-redaction | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: 422 bodies keep the LLM key out (`F04.create-validation-redaction`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-05-422-bodies-keep-the-llm-key-out-f04-crea.json | +| F04.create-worktree | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Git worktree (`F04.create-worktree`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-09-git-worktree-f04-create-worktree.json | +| F04.delete | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Delete (`F04.delete`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-29-delete-f04-delete.json | +| F04.delete-unknown-id | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: DELETE on an unknown id (`F04.delete-unknown-id`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-30-delete-on-an-unknown-id-f04-delete-unkno.json | +| F04.final-response | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Final response (`F04.final-response`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-16-final-response-f04-final-response.json | +| F04.get | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Read one conversation (`F04.get`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-18-read-one-conversation-f04-get.json | +| F04.patch-errors | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: PATCH errors (`F04.patch-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-26-patch-errors-f04-patch-errors.json | +| F04.patch-tags | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Retag (`F04.patch-tags`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-25-retag-f04-patch-tags.json | +| F04.patch-title | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Rename (`F04.patch-title`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-24-rename-f04-patch-title.json | +| F04.patch-unknown-id | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: PATCH on an unknown id (`F04.patch-unknown-id`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-28-patch-on-an-unknown-id-f04-patch-unknown.json | +| F04.patch-whitespace-title | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Whitespace-only title (`F04.patch-whitespace-title`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-27-whitespace-only-title-f04-patch-whitespa.json | +| F04.read-no-hydrate | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Cold reads do not load a conversation (`F04.read-no-hydrate`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-32-cold-reads-do-not-load-a-conversation-f0.json | +| F04.restart-persist | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Restart (`F04.restart-persist`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-31-restart-f04-restart-persist.json | +| F04.run-real | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: A real run (`F04.run-real`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-15-a-real-run-f04-run-real.json | +| F04.search-filter | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Status filter (`F04.search-filter`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-22-status-filter-f04-search-filter.json | +| F04.search-page | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Search pages (`F04.search-page`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-21-search-pages-f04-search-page.json | +| F04.title-llm-profile | 20261111-082303-ef07-map-f04 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F04-conversation-lifecycle.md: Title from a saved LLM profile (`F04.title-llm-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-ef07-map-f04/evidence/map-run/F04-conversation-lifecycle-17-title-from-a-saved-llm-profile-f04-title.json | +| F05.control-errors | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Auth and malformed ids (`F05.control-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-13-auth-and-malformed-ids-f05-control-error.json | +| F05.interrupt-idle | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Interrupt with nothing running (`F05.interrupt-idle`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-07-interrupt-with-nothing-running-f05-inter.json | +| F05.interrupt-running | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Interrupt a running conversation (`F05.interrupt-running`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-18-interrupt-a-running-conversation-f05-int.json | +| F05.interrupt-tool-call | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Interrupt during a tool call (`F05.interrupt-tool-call`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-21-interrupt-during-a-tool-call-f05-interru.json | +| F05.max-iterations-resume | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Resume after the cap (`F05.max-iterations-resume`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-27-resume-after-the-cap-f05-max-iterations-.json | +| F05.max-iterations-stop | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Iteration cap (`F05.max-iterations-stop`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-26-iteration-cap-f05-max-iterations-stop.json | +| F05.pause-idle | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Pause an idle conversation (`F05.pause-idle`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-04-pause-an-idle-conversation-f05-pause-idl.json | +| F05.pause-repeat | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Pause twice (`F05.pause-repeat`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-05-pause-twice-f05-pause-repeat.json | +| F05.pause-running | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Pause during a long model call (`F05.pause-running`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-15-pause-during-a-long-model-call-f05-pause.json | +| F05.pause-running-release | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Release a paused step with interrupt (`F05.pause-running-release`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-17-release-a-paused-step-with-interrupt-f05.json | +| F05.pause-terminal | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Pause and interrupt a terminal conversation (`F05.pause-terminal`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-09-pause-and-interrupt-a-terminal-conversat.json | +| F05.pause-tool-step | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Pause during a tool call (`F05.pause-tool-step`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-20-pause-during-a-tool-call-f05-pause-tool-.json | +| F05.restart-hard-running | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Crash while running (`F05.restart-hard-running`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-30-crash-while-running-f05-restart-hard-run.json | +| F05.restart-keeps-status | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Statuses survive the restart (`F05.restart-keeps-status`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-29-statuses-survive-the-restart-f05-restart.json | +| F05.restart-running-graceful | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Graceful restart while running (`F05.restart-running-graceful`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-28-graceful-restart-while-running-f05-resta.json | +| F05.run-after-interrupt | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Resume after an interrupt (`F05.run-after-interrupt`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-19-resume-after-an-interrupt-f05-run-after-.json | +| F05.run-capacity | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Run capacity (`F05.run-capacity`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-22-run-capacity-f05-run-capacity.json | +| F05.run-conflict | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Run while running (`F05.run-conflict`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-14-run-while-running-f05-run-conflict.json | +| F05.run-error | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: A run that fails (`F05.run-error`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-08-a-run-that-fails-f05-run-error.json | +| F05.run-error-responsive | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Responsive while a run fails (`F05.run-error-responsive`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-10-responsive-while-a-run-fails-f05-run-err.json | +| F05.run-finished-noop | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Run with nothing to do (`F05.run-finished-noop`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-02-run-with-nothing-to-do-f05-run-finished-.json | +| F05.run-new-input | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: New input after finishing (`F05.run-new-input`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-03-new-input-after-finishing-f05-run-new-in.json | +| F05.run-queued | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Run queued input on a real model (`F05.run-queued`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-01-run-queued-input-on-a-real-model-f05-run.json | +| F05.run-resume-paused | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Resume a paused conversation (`F05.run-resume-paused`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-06-resume-a-paused-conversation-f05-run-res.json | +| F05.stuck-detected | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: A run that loops ends `stuck` (`F05.stuck-detected`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-23-a-run-that-loops-ends-stuck-f05-stuck-de.json | +| F05.stuck-new-input | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: New input un-sticks it (`F05.stuck-new-input`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-25-new-input-un-sticks-it-f05-stuck-new-inp.json | +| F05.stuck-rerun | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Run a stuck conversation again (`F05.stuck-rerun`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-24-run-a-stuck-conversation-again-f05-stuck.json | +| F05.unknown-id-pause | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Pause or interrupt an unknown id (`F05.unknown-id-pause`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-12-pause-or-interrupt-an-unknown-id-f05-unk.json | +| F05.unknown-id-run | 20261111-082303-c7a0-map-f05 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F05-run-pause-interrupt.md: Run an unknown id (`F05.unknown-id-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-c7a0-map-f05/evidence/map-run/F05-run-pause-interrupt-11-run-an-unknown-id-f05-unknown-id-run.json | +| F06.batch-get | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Batch get (`F06.batch-get`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-10-batch-get-f06-batch-get.json | +| F06.count | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Count (`F06.count`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-07-count-f06-count.json | +| F06.get-one | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Get one event (`F06.get-one`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-08-get-one-event-f06-get-one.json | +| F06.not-found-auth | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Unknown conversation, bad id, no key (`F06.not-found-auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-11-unknown-conversation-bad-id-no-key-f06-n.json | +| F06.restart-persist | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: History survives a restart (`F06.restart-persist`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-25-history-survives-a-restart-f06-restart-p.json | +| F06.search-filters | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Filters (`F06.search-filters`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-04-filters-f06-search-filters.json | +| F06.search-kind-short | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Short kind names (`F06.search-kind-short`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-05-short-kind-names-f06-search-kind-short.json | +| F06.search-page | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Paging and sort order (`F06.search-page`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-02-paging-and-sort-order-f06-search-page.json | +| F06.search-time-window | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Time window (`F06.search-time-window`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-06-time-window-f06-search-time-window.json | +| F06.search-timestamp-order | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Log order is timestamp order (`F06.search-timestamp-order`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-23-log-order-is-timestamp-order-f06-search-.json | +| F06.search-validation | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Bad search parameters (`F06.search-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-03-bad-search-parameters-f06-search-validat.json | +| F06.send-image | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Image content (`F06.send-image`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-13-image-content-f06-send-image.json | +| F06.send-no-run | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Append a message without running (`F06.send-no-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-01-append-a-message-without-running-f06-sen.json | +| F06.send-resets-finished | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: A message reopens a finished conversation (`F06.send-resets-finished`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-24-a-message-reopens-a-finished-conversatio.json | +| F06.send-run | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Run a turn and watch it live (`F06.send-run`, `F06.ws-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-22-run-a-turn-and-watch-it-live-f06-send-ru.json | +| F06.send-run-limit | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Run capacity full (`F06.send-run-limit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-26-run-capacity-full-f06-send-run-limit.json | +| F06.send-validation | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Message validation (`F06.send-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-12-message-validation-f06-send-validation.json | +| F06.send-while-running | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: A message while the agent runs (`F06.send-while-running`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-27-a-message-while-the-agent-runs-f06-send-.json | +| F06.unknown-event | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Unknown event id (`F06.unknown-event`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-09-unknown-event-id-f06-unknown-event.json | +| F06.ws-bad-frame | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Bad frames on the socket (`F06.ws-bad-frame`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-20-bad-frames-on-the-socket-f06-ws-bad-fram.json | +| F06.ws-handshake | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Handshake: auth and close codes (`F06.ws-handshake`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-16-handshake-auth-and-close-codes-f06-ws-ha.json | +| F06.ws-live | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Run a turn and watch it live (`F06.send-run`, `F06.ws-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-22-run-a-turn-and-watch-it-live-f06-send-ru.json | +| F06.ws-replay-all | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Replay everything (`F06.ws-replay-all`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-18-replay-everything-f06-ws-replay-all.json | +| F06.ws-replay-since | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Replay since a timestamp (`F06.ws-replay-since`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-19-replay-since-a-timestamp-f06-ws-replay-s.json | +| F06.ws-send | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Send a message over the socket (`F06.ws-send`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-21-send-a-message-over-the-socket-f06-ws-se.json | +| F06.ws-snapshot | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: First frame is a state snapshot (`F06.ws-snapshot`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-15-first-frame-is-a-state-snapshot-f06-ws-s.json | +| F06.ws-streaming-deltas | 20261111-082303-2fb6-map-f06 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F06-conversation-events.md: Streamed deltas are transient (`F06.ws-streaming-deltas`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082303-2fb6-map-f06/evidence/map-run/F06-conversation-events-28-streamed-deltas-are-transient-f06-ws-str.json | +| F07.auth | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Authentication (`F07.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-10-authentication-f07-auth.json | +| F07.auth-frame-ignored | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Redundant auth frame (`F07.auth-frame-ignored`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-09-redundant-auth-frame-f07-auth-frame-igno.json | +| F07.bad-handshake | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Malformed handshake (`F07.bad-handshake`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-12-malformed-handshake-f07-bad-handshake.json | +| F07.error-frame | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Invalid message (`F07.error-frame`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-06-invalid-message-f07-error-frame.json | +| F07.error-non-json | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Non-JSON and binary frames (`F07.error-non-json`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-07-non-json-and-binary-frames-f07-error-non.json | +| F07.frame-too-large | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Oversized event (`F07.frame-too-large`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-21-oversized-event-f07-frame-too-large.json | +| F07.full-replay | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Full replay (`F07.full-replay`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-03-full-replay-f07-full-replay.json | +| F07.item-aborted | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Interrupted stream (`F07.item-aborted`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-20-interrupted-stream-f07-item-aborted.json | +| F07.live-durable | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Live durable frame (`F07.live-durable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-13-live-durable-frame-f07-live-durable.json | +| F07.live-order | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Live frames in seq order (`F07.live-order`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-14-live-frames-in-seq-order-f07-live-order.json | +| F07.progress-frames | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Streaming progress frames (`F07.progress-frames`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-18-streaming-progress-frames-f07-progress-f.json | +| F07.progress-not-replayed | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Progress is never replayed (`F07.progress-not-replayed`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-19-progress-is-never-replayed-f07-progress-.json | +| F07.reject-non-user | 20261111-083109-0c87-map-f07 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F07-session-socket.md: Non-user role (`F07.reject-non-user`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083109-0c87-map-f07/evidence/map-run/F07-session-socket-01-non-user-role-f07-reject-non-user.json | +| F07.reject-non-user | 20261111-083138-fed1-map-f07 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F07-session-socket.md: Non-user role (`F07.reject-non-user`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083138-fed1-map-f07/evidence/map-run/F07-session-socket-01-non-user-role-f07-reject-non-user.json | +| F07.replay-then-live | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Replay and live tail, each event once (`F07.replay-then-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-15-replay-and-live-tail-each-event-once-f07.json | +| F07.restart-resume | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Cursor survives a restart (`F07.restart-resume`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-17-cursor-survives-a-restart-f07-restart-re.json | +| F07.resume-cursor | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Resume from a cursor (`F07.resume-cursor`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-05-resume-from-a-cursor-f07-resume-cursor.json | +| F07.send-message | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Send a message on the socket (`F07.send-message`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-16-send-a-message-on-the-socket-f07-send-me.json | +| F07.seq-on-disk | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Seq is the on-disk index (`F07.seq-on-disk`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-04-seq-is-the-on-disk-index-f07-seq-on-disk.json | +| F07.slow-consumer | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Stalled reader (`F07.slow-consumer`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-22-stalled-reader-f07-slow-consumer.json | +| F07.snapshot-transient | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: State snapshot (`F07.snapshot-transient`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-02-state-snapshot-f07-snapshot-transient.json | +| F07.subscriber-limit | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Subscriber cap (`F07.subscriber-limit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-23-subscriber-cap-f07-subscriber-limit.json | +| F07.sync-first | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Sync comes first (`F07.sync-first`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-01-sync-comes-first-f07-sync-first.json | +| F07.unknown-conversation | 20261111-082602-7f35-map-f07 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F07-session-socket.md: Unknown conversation (`F07.unknown-conversation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082602-7f35-map-f07/evidence/map-run/F07-session-socket-11-unknown-conversation-f07-unknown-convers.json | +| F08.always-confirm-wait | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Wait for confirmation (`F08.always-confirm-wait`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-03-wait-for-confirmation-f08-always-confirm.json | +| F08.analyzer-errors | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Analyzer errors (`F08.analyzer-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-10-analyzer-errors-f08-analyzer-errors.json | +| F08.analyzer-set-clear | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Set and clear the analyzer (`F08.analyzer-set-clear`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-09-set-and-clear-the-analyzer-f08-analyzer-.json | +| F08.confirm-accept | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Accept (`F08.confirm-accept`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-04-accept-f08-confirm-accept.json | +| F08.confirm-capacity | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Accept with the run capacity full (`F08.confirm-capacity`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-07-accept-with-the-run-capacity-full-f08-co.json | +| F08.confirm-errors | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Confirmation errors (`F08.confirm-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-06-confirmation-errors-f08-confirm-errors.json | +| F08.confirm-real | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: A real model asks first (`F08.confirm-real`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-08-a-real-model-asks-first-f08-confirm-real.json | +| F08.confirm-reject | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Reject (`F08.confirm-reject`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-05-reject-f08-confirm-reject.json | +| F08.confirm-risky | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Risk-based confirmation (`F08.confirm-risky`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-11-risk-based-confirmation-f08-confirm-risk.json | +| F08.policy-errors | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Policy errors (`F08.policy-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-02-policy-errors-f08-policy-errors.json | +| F08.policy-set | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Set the confirmation policy (`F08.policy-set`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-01-set-the-confirmation-policy-f08-policy-s.json | +| F08.secrets-bad-value | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Non-string secret value (`F08.secrets-bad-value`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-15-non-string-secret-value-f08-secrets-bad-.json | +| F08.secrets-errors | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Secret errors (`F08.secrets-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-14-secret-errors-f08-secrets-errors.json | +| F08.secrets-lookup | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Lookup secret (`F08.secrets-lookup`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-18-lookup-secret-f08-secrets-lookup.json | +| F08.secrets-lookup-self | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Lookup of the server's own secret (`F08.secrets-lookup-self`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-19-lookup-of-the-server-s-own-secret-f08-se.json | +| F08.secrets-no-cipher | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: No cipher (`F08.secrets-no-cipher`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-21-no-cipher-f08-secrets-no-cipher.json | +| F08.secrets-override | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Override a start-time secret (`F08.secrets-override`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-17-override-a-start-time-secret-f08-secrets.json | +| F08.secrets-profile-scope | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Agent Profile secret scope (`F08.secrets-profile-scope`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-20-agent-profile-secret-scope-f08-secrets-p.json | +| F08.secrets-restart | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Restart with a saved secret (`F08.secrets-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-22-restart-with-a-saved-secret-f08-secrets-.json | +| F08.secrets-terminal | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Secrets in the terminal (`F08.secrets-terminal`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-16-secrets-in-the-terminal-f08-secrets-term.json | +| F08.secrets-update | 20261111-082612-66f0-map-f08 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F08-confirmation-security-secrets.md: Update secrets (`F08.secrets-update`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082612-66f0-map-f08/evidence/map-run/F08-confirmation-security-secrets-12-update-secrets-f08-secrets-update.json | +| F09.acp-effort-deferred | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: ACP effort before the first run (`F09.acp-effort-deferred`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-22-acp-effort-before-the-first-run-f09-acp-.json | +| F09.acp-effort-deferred | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: ACP effort before the first run (`F09.acp-effort-deferred`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-22-acp-effort-before-the-first-run-f09-acp-.json | +| F09.acp-effort-live | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Live effort switch (`F09.acp-effort-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-24-live-effort-switch-f09-acp-effort-live.json | +| F09.acp-effort-live | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Live effort switch (`F09.acp-effort-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-24-live-effort-switch-f09-acp-effort-live.json | +| F09.acp-effort-live-timeout | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Live effort switch that times out (`F09.acp-effort-live-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-27-live-effort-switch-that-times-out-f09-ac.json | +| F09.acp-effort-live-timeout | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Live effort switch that times out (`F09.acp-effort-live-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-27-live-effort-switch-that-times-out-f09-ac.json | +| F09.acp-effort-model-switch | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Effort across a model switch (`F09.acp-effort-model-switch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-26-effort-across-a-model-switch-f09-acp-eff.json | +| F09.acp-effort-model-switch | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Effort across a model switch (`F09.acp-effort-model-switch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-26-effort-across-a-model-switch-f09-acp-eff.json | +| F09.acp-effort-native | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: ACP effort on a regular conversation (`F09.acp-effort-native`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-21-acp-effort-on-a-regular-conversation-f09.json | +| F09.acp-effort-native | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: ACP effort on a regular conversation (`F09.acp-effort-native`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-21-acp-effort-on-a-regular-conversation-f09.json | +| F09.acp-effort-unoffered | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Unoffered effort on a live session (`F09.acp-effort-unoffered`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-25-unoffered-effort-on-a-live-session-f09-a.json | +| F09.acp-effort-unoffered | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Unoffered effort on a live session (`F09.acp-effort-unoffered`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-25-unoffered-effort-on-a-live-session-f09-a.json | +| F09.acp-model-blank | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Blank ACP model (`F09.acp-model-blank`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-16-blank-acp-model-f09-acp-model-blank.json | +| F09.acp-model-blank | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Blank ACP model (`F09.acp-model-blank`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-16-blank-acp-model-f09-acp-model-blank.json | +| F09.acp-model-blank | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Blank ACP model (`F09.acp-model-blank`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-16-blank-acp-model-f09-acp-model-blank.json | +| F09.acp-model-deferred | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: ACP model before the first run (`F09.acp-model-deferred`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-15-acp-model-before-the-first-run-f09-acp-m.json | +| F09.acp-model-deferred | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: ACP model before the first run (`F09.acp-model-deferred`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-15-acp-model-before-the-first-run-f09-acp-m.json | +| F09.acp-model-deferred | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: ACP model before the first run (`F09.acp-model-deferred`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-15-acp-model-before-the-first-run-f09-acp-m.json | +| F09.acp-model-id-bare | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Bare model id with a legacy effort suffix (`F09.acp-model-id-bare`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-23-bare-model-id-with-a-legacy-effort-suffi.json | +| F09.acp-model-id-bare | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Bare model id with a legacy effort suffix (`F09.acp-model-id-bare`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-23-bare-model-id-with-a-legacy-effort-suffi.json | +| F09.acp-model-live | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Live ACP switch (`F09.acp-model-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-18-live-acp-switch-f09-acp-model-live.json | +| F09.acp-model-live | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Live ACP switch (`F09.acp-model-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-18-live-acp-switch-f09-acp-model-live.json | +| F09.acp-model-live | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Live ACP switch (`F09.acp-model-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-18-live-acp-switch-f09-acp-model-live.json | +| F09.acp-model-live-timeout | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Live switch that times out (`F09.acp-model-live-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-20-live-switch-that-times-out-f09-acp-model.json | +| F09.acp-model-live-timeout | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Live switch that times out (`F09.acp-model-live-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-20-live-switch-that-times-out-f09-acp-model.json | +| F09.acp-model-live-timeout | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Live switch that times out (`F09.acp-model-live-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-20-live-switch-that-times-out-f09-acp-model.json | +| F09.acp-model-native | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: ACP switch on a regular conversation (`F09.acp-model-native`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-14-acp-switch-on-a-regular-conversation-f09.json | +| F09.acp-model-native | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: ACP switch on a regular conversation (`F09.acp-model-native`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-14-acp-switch-on-a-regular-conversation-f09.json | +| F09.acp-model-native | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: ACP switch on a regular conversation (`F09.acp-model-native`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-14-acp-switch-on-a-regular-conversation-f09.json | +| F09.auth-required | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Key required (`F09.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-01-key-required-f09-auth-required.json | +| F09.auth-required | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Key required (`F09.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-01-key-required-f09-auth-required.json | +| F09.auth-required | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Key required (`F09.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-01-key-required-f09-auth-required.json | +| F09.load-plugin | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Load a plugin (`F09.load-plugin`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-23-load-a-plugin-f09-load-plugin.json | +| F09.load-plugin | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Load a plugin (`F09.load-plugin`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-32-load-a-plugin-f09-load-plugin.json | +| F09.load-plugin | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Load a plugin (`F09.load-plugin`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-32-load-a-plugin-f09-load-plugin.json | +| F09.load-plugin-errors | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Plugin load errors (`F09.load-plugin-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-25-plugin-load-errors-f09-load-plugin-error.json | +| F09.load-plugin-errors | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Plugin load errors (`F09.load-plugin-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-34-plugin-load-errors-f09-load-plugin-error.json | +| F09.load-plugin-errors | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Plugin load errors (`F09.load-plugin-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-34-plugin-load-errors-f09-load-plugin-error.json | +| F09.load-plugin-hook-runs | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: The loaded hook runs (`F09.load-plugin-hook-runs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-24-the-loaded-hook-runs-f09-load-plugin-hoo.json | +| F09.load-plugin-hook-runs | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: The loaded hook runs (`F09.load-plugin-hook-runs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-33-the-loaded-hook-runs-f09-load-plugin-hoo.json | +| F09.load-plugin-hook-runs | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: The loaded hook runs (`F09.load-plugin-hook-runs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-33-the-loaded-hook-runs-f09-load-plugin-hoo.json | +| F09.load-plugin-sdk | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Load a plugin from the Python SDK (`F09.load-plugin-sdk`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-27-load-a-plugin-from-the-python-sdk-f09-lo.json | +| F09.load-plugin-sdk | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Load a plugin from the Python SDK (`F09.load-plugin-sdk`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-36-load-a-plugin-from-the-python-sdk-f09-lo.json | +| F09.load-plugin-sdk | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Load a plugin from the Python SDK (`F09.load-plugin-sdk`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-36-load-a-plugin-from-the-python-sdk-f09-lo.json | +| F09.restart-persist | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Restart (`F09.restart-persist`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-28-restart-f09-restart-persist.json | +| F09.restart-persist | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Restart (`F09.restart-persist`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-37-restart-f09-restart-persist.json | +| F09.restart-persist | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Restart (`F09.restart-persist`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-37-restart-f09-restart-persist.json | +| F09.switch-llm | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Switch to a caller-supplied LLM (`F09.switch-llm`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-02-switch-to-a-caller-supplied-llm-f09-swit.json | +| F09.switch-llm | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Switch to a caller-supplied LLM (`F09.switch-llm`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-02-switch-to-a-caller-supplied-llm-f09-swit.json | +| F09.switch-llm | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Switch to a caller-supplied LLM (`F09.switch-llm`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-02-switch-to-a-caller-supplied-llm-f09-swit.json | +| F09.switch-llm-acp-refused | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: LLM switches on an ACP conversation (`F09.switch-llm-acp-refused`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-17-llm-switches-on-an-acp-conversation-f09-.json | +| F09.switch-llm-acp-refused | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: LLM switches on an ACP conversation (`F09.switch-llm-acp-refused`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-17-llm-switches-on-an-acp-conversation-f09-.json | +| F09.switch-llm-acp-refused | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: LLM switches on an ACP conversation (`F09.switch-llm-acp-refused`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-17-llm-switches-on-an-acp-conversation-f09-.json | +| F09.switch-llm-encrypted-key | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Mid-conversation switch with a cipher token (`F09.switch-llm-encrypted-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-05-mid-conversation-switch-with-a-cipher-to.json | +| F09.switch-llm-encrypted-key | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Mid-conversation switch with a cipher token (`F09.switch-llm-encrypted-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-05-mid-conversation-switch-with-a-cipher-to.json | +| F09.switch-llm-encrypted-key | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Mid-conversation switch with a cipher token (`F09.switch-llm-encrypted-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-05-mid-conversation-switch-with-a-cipher-to.json | +| F09.switch-llm-errors | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Switch errors (`F09.switch-llm-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-04-switch-errors-f09-switch-llm-errors.json | +| F09.switch-llm-errors | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Switch errors (`F09.switch-llm-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-04-switch-errors-f09-switch-llm-errors.json | +| F09.switch-llm-errors | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Switch errors (`F09.switch-llm-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-04-switch-errors-f09-switch-llm-errors.json | +| F09.switch-llm-mid-run | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Switch during a running turn (`F09.switch-llm-mid-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-12-switch-during-a-running-turn-f09-switch-.json | +| F09.switch-llm-mid-run | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Switch during a running turn (`F09.switch-llm-mid-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-12-switch-during-a-running-turn-f09-switch-.json | +| F09.switch-llm-mid-run | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Switch during a running turn (`F09.switch-llm-mid-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-12-switch-during-a-running-turn-f09-switch-.json | +| F09.switch-llm-provider-connection | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Switch through a provider connection (`F09.switch-llm-provider-connection`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-06-switch-through-a-provider-connection-f09.json | +| F09.switch-llm-provider-connection | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Switch through a provider connection (`F09.switch-llm-provider-connection`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-06-switch-through-a-provider-connection-f09.json | +| F09.switch-llm-provider-connection | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Switch through a provider connection (`F09.switch-llm-provider-connection`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-06-switch-through-a-provider-connection-f09.json | +| F09.switch-llm-registered-usage-id | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Switch onto a registered usage id (`F09.switch-llm-registered-usage-id`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-10-switch-onto-a-registered-usage-id-f09-sw.json | +| F09.switch-llm-registered-usage-id | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Switch onto a registered usage id (`F09.switch-llm-registered-usage-id`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-10-switch-onto-a-registered-usage-id-f09-sw.json | +| F09.switch-llm-registered-usage-id | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Switch onto a registered usage id (`F09.switch-llm-registered-usage-id`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-10-switch-onto-a-registered-usage-id-f09-sw.json | +| F09.switch-llm-ws | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Socket view of a switch (`F09.switch-llm-ws`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-03-socket-view-of-a-switch-f09-switch-llm-w.json | +| F09.switch-llm-ws | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Socket view of a switch (`F09.switch-llm-ws`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-03-socket-view-of-a-switch-f09-switch-llm-w.json | +| F09.switch-llm-ws | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Socket view of a switch (`F09.switch-llm-ws`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-03-socket-view-of-a-switch-f09-switch-llm-w.json | +| F09.switch-profile | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Switch to a saved profile (`F09.switch-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-07-switch-to-a-saved-profile-f09-switch-pro.json | +| F09.switch-profile | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Switch to a saved profile (`F09.switch-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-07-switch-to-a-saved-profile-f09-switch-pro.json | +| F09.switch-profile | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Switch to a saved profile (`F09.switch-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-07-switch-to-a-saved-profile-f09-switch-pro.json | +| F09.switch-profile-errors | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: Profile switch errors (`F09.switch-profile-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-08-profile-switch-errors-f09-switch-profile.json | +| F09.switch-profile-errors | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: Profile switch errors (`F09.switch-profile-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-08-profile-switch-errors-f09-switch-profile.json | +| F09.switch-profile-errors | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: Profile switch errors (`F09.switch-profile-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-08-profile-switch-errors-f09-switch-profile.json | +| F09.ts-client | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: TypeScript client (`F09.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-21-typescript-client-f09-ts-client.json | +| F09.ts-client | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: TypeScript client (`F09.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-29-typescript-client-f09-ts-client.json | +| F09.ts-client | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: TypeScript client (`F09.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-29-typescript-client-f09-ts-client.json | +| F09.ts-client-acp-docs | 20261111-082653-db52-map-f09 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F09-model-switching-plugins.md: TypeScript docstrings for a pre-run ACP switch (`F09.ts-client-acp-docs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082653-db52-map-f09/evidence/map-run/F09-model-switching-plugins-22-typescript-docstrings-for-a-pre-run-acp-.json | +| F09.ts-client-acp-docs | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: TypeScript docstrings for a pre-run ACP switch (`F09.ts-client-acp-docs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-30-typescript-docstrings-for-a-pre-run-acp-.json | +| F09.ts-client-acp-docs | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: TypeScript docstrings for a pre-run ACP switch (`F09.ts-client-acp-docs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-30-typescript-docstrings-for-a-pre-run-acp-.json | +| F09.ts-client-acp-effort | 20261111-083508-5969-map-f09 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F09-model-switching-plugins.md: TypeScript effort switch (`F09.ts-client-acp-effort`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083508-5969-map-f09/evidence/map-run/F09-model-switching-plugins-31-typescript-effort-switch-f09-ts-client-a.json | +| F09.ts-client-acp-effort | 20261111-084052-4ce1-map-f09 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F09-model-switching-plugins.md: TypeScript effort switch (`F09.ts-client-acp-effort`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084052-4ce1-map-f09/evidence/map-run/F09-model-switching-plugins-31-typescript-effort-switch-f09-ts-client-a.json | +| F10.ask-agent | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Ask a side question (`F10.ask-agent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-20-ask-a-side-question-f10-ask-agent.json | +| F10.ask-agent-errors | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Bad input, no key, unreachable model (`F10.ask-agent-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-21-bad-input-no-key-unreachable-model-f10-a.json | +| F10.ask-agent-running | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Side question during a run (`F10.ask-agent-running`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-22-side-question-during-a-run-f10-ask-agent.json | +| F10.ask-agent-unknown | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Unknown conversation (`F10.ask-agent-unknown`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-23-unknown-conversation-f10-ask-agent-unkno.json | +| F10.condense-errors | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Unknown, malformed and unauthenticated (`F10.condense-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-19-unknown-malformed-and-unauthenticated-f1.json | +| F10.condense-llm | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Condense with DeepSeek (`F10.condense-llm`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-15-condense-with-deepseek-f10-condense-llm.json | +| F10.condense-llm-failure | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Unreachable condenser model (`F10.condense-llm-failure`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-18-unreachable-condenser-model-f10-condense.json | +| F10.condense-no-condenser | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: No request-handling condenser (`F10.condense-no-condenser`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-16-no-request-handling-condenser-f10-conden.json | +| F10.condense-status | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Refusal status (`F10.condense-status`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-17-refusal-status-f10-condense-status.json | +| F10.fork-errors | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Unknown, taken and invalid input (`F10.fork-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-06-unknown-taken-and-invalid-input-f10-fork.json | +| F10.fork-from-event | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Fork from an event (`F10.fork-from-event`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-03-fork-from-an-event-f10-fork-from-event.json | +| F10.fork-independent | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Fork and source diverge (`F10.fork-independent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-05-fork-and-source-diverge-f10-fork-indepen.json | +| F10.fork-metrics | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Metrics on a fork (`F10.fork-metrics`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-04-metrics-on-a-fork-f10-fork-metrics.json | +| F10.fork-named | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Named fork (`F10.fork-named`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-02-named-fork-f10-fork-named.json | +| F10.fork-tags-consistent | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Fork tags agree with the stored metadata (`F10.fork-tags-consistent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-07-fork-tags-agree-with-the-stored-metadata.json | +| F10.fork-whole | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Fork the whole log (`F10.fork-whole`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-01-fork-the-whole-log-f10-fork-whole.json | +| F10.navigate-branch | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: The next event branches (`F10.navigate-branch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-11-the-next-event-branches-f10-navigate-bra.json | +| F10.navigate-empty | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Navigate to the empty tree (`F10.navigate-empty`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-12-navigate-to-the-empty-tree-f10-navigate-.json | +| F10.navigate-errors | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Unknown, missing and unauthenticated (`F10.navigate-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-13-unknown-missing-and-unauthenticated-f10-.json | +| F10.navigate-event | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Navigate to an earlier event (`F10.navigate-event`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-10-navigate-to-an-earlier-event-f10-navigat.json | +| F10.navigate-final-response | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Final response after navigating back (`F10.navigate-final-response`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-14-final-response-after-navigating-back-f10.json | +| F10.restart-cold-calls | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: Condense and ask on a cold conversation (`F10.restart-cold-calls`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-25-condense-and-ask-on-a-cold-conversation-.json | +| F10.restart-persist | 20261111-082853-9a36-map-f10 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F10-fork-navigate-condense-ask.md: State after a restart (`F10.restart-persist`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082853-9a36-map-f10/evidence/map-run/F10-fork-navigate-condense-ask-24-state-after-a-restart-f10-restart-persis.json | +| F11.goal-auth | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Authentication (`F11.goal-auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-03-authentication-f11-goal-auth.json | +| F11.goal-busy | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Busy while a round is in flight (`F11.goal-busy`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-12-busy-while-a-round-is-in-flight-f11-goal.json | +| F11.goal-busy-run | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Busy with an ordinary run (`F11.goal-busy-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-17-busy-with-an-ordinary-run-f11-goal-busy-.json | +| F11.goal-cap-overrun | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: A resume after a failed last audit stays within the cap (`F11.goal-cap-overrun`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-24-a-resume-after-a-failed-last-audit-stays.json | +| F11.goal-capacity | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Capacity (`F11.goal-capacity`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-22-capacity-f11-goal-capacity.json | +| F11.goal-capped | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Round cap (`F11.goal-capped`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-10-round-cap-f11-goal-capped.json | +| F11.goal-complete | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: A goal on a real model (`F11.goal-complete`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-06-a-goal-on-a-real-model-f11-goal-complete.json | +| F11.goal-delete-active | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Delete a conversation with an active loop (`F11.goal-delete-active`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-23-delete-a-conversation-with-an-active-loo.json | +| F11.goal-followup | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Incomplete verdict re-prompts (`F11.goal-followup`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-09-incomplete-verdict-re-prompts-f11-goal-f.json | +| F11.goal-in-place | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Same conversation, no new one (`F11.goal-in-place`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-08-same-conversation-no-new-one-f11-goal-in.json | +| F11.goal-interrupt-run | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Interrupt the round (`F11.goal-interrupt-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-16-interrupt-the-round-f11-goal-interrupt-r.json | +| F11.goal-judge-error | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: The judge fails (`F11.goal-judge-error`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-21-the-judge-fails-f11-goal-judge-error.json | +| F11.goal-restart-active | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: A loop active at a restart ends "interrupted" (`F11.goal-restart-active`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-25-a-loop-active-at-a-restart-ends-interrup.json | +| F11.goal-restart-resume | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Resume after a restart (`F11.goal-restart-resume`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-19-resume-after-a-restart-f11-goal-restart-.json | +| F11.goal-resume | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Resume a stopped loop (`F11.goal-resume`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-18-resume-a-stopped-loop-f11-goal-resume.json | +| F11.goal-resume-finished | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Nothing to resume after the end (`F11.goal-resume-finished`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-11-nothing-to-resume-after-the-end-f11-goal.json | +| F11.goal-resume-none | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Resume with no goal (`F11.goal-resume-none`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-05-resume-with-no-goal-f11-goal-resume-none.json | +| F11.goal-run-error | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: The round's provider fails (`F11.goal-run-error`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-20-the-round-s-provider-fails-f11-goal-run-.json | +| F11.goal-stop | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Stop an active loop (`F11.goal-stop`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-13-stop-an-active-loop-f11-goal-stop.json | +| F11.goal-stop-noop | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Stop with nothing to stop (`F11.goal-stop-noop`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-04-stop-with-nothing-to-stop-f11-goal-stop-.json | +| F11.goal-superseded | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: A user message supersedes the loop (`F11.goal-superseded`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-14-a-user-message-supersedes-the-loop-f11-g.json | +| F11.goal-superseded-socket | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: A socket message supersedes the loop (`F11.goal-superseded-socket`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-15-a-socket-message-supersedes-the-loop-f11.json | +| F11.goal-unknown-id | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Unknown and malformed ids (`F11.goal-unknown-id`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-02-unknown-and-malformed-ids-f11-goal-unkno.json | +| F11.goal-validation | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Invalid goal requests (`F11.goal-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-01-invalid-goal-requests-f11-goal-validatio.json | +| F11.goal-ws | 20261111-082952-e27e-map-f11 | 5ff99f1b6ca28f7d16870b69f57a31351e59bcd0 | pass | map run F11-goals.md: Goal status on the socket (`F11.goal-ws`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-082952-e27e-map-f11/evidence/map-run/F11-goals-07-goal-status-on-the-socket-f11-goal-ws.json | +| F12.binding-live-acp | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Bind a loaded ACP Codex conversation (`F12.binding-live-acp`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-18-bind-a-loaded-acp-codex-conversation-f12.json | +| F12.binding-native-cold | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Regular conversation, not loaded (`F12.binding-native-cold`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-17-regular-conversation-not-loaded-f12-bind.json | +| F12.binding-native-too-late | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Regular conversation, loaded (`F12.binding-native-too-late`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-16-regular-conversation-loaded-f12-binding-.json | +| F12.binding-pending | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Pending binding for an unknown id (`F12.binding-pending`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-13-pending-binding-for-an-unknown-id-f12-bi.json | +| F12.binding-pending-cleared | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: A pause drops pending bindings (`F12.binding-pending-cleared`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-23-a-pause-drops-pending-bindings-f12-bindi.json | +| F12.binding-pending-on-create | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Pending binding consumed at create (`F12.binding-pending-on-create`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-22-pending-binding-consumed-at-create-f12-b.json | +| F12.binding-profile-scope | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Profile excludes the secret (`F12.binding-profile-scope`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-24-profile-excludes-the-secret-f12-binding-.json | +| F12.binding-rebind | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Re-bind (`F12.binding-rebind`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-19-re-bind-f12-binding-rebind.json | +| F12.binding-required-after-pause | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Required after a pause (`F12.binding-required-after-pause`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-20-required-after-a-pause-f12-binding-requi.json | +| F12.binding-required-after-restart | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Required after a restart (`F12.binding-required-after-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-21-required-after-a-restart-f12-binding-req.json | +| F12.binding-source-statuses | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Source answers (`F12.binding-source-statuses`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-14-source-answers-f12-binding-source-status.json | +| F12.binding-source-unavailable | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Unavailable source (`F12.binding-source-unavailable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-15-unavailable-source-f12-binding-source-un.json | +| F12.binding-unsupported-secret | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Unsupported secret names (`F12.binding-unsupported-secret`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-11-unsupported-secret-names-f12-binding-uns.json | +| F12.binding-validation | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Body validation (`F12.binding-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-12-body-validation-f12-binding-validation.json | +| F12.discovery | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Discovery (`F12.discovery`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-01-discovery-f12-discovery.json | +| F12.docker-runtime-contract | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Docker runtime mode without a daemon (`F12.docker-runtime-contract`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-29-docker-runtime-mode-without-a-daemon-f12.json | +| F12.managed-key-refresh | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Managed key refresh on a 401 (`F12.managed-key-refresh`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-26-managed-key-refresh-on-a-401-f12-managed.json | +| F12.managed-key-scope | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Refresh scope (`F12.managed-key-scope`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-27-refresh-scope-f12-managed-key-scope.json | +| F12.managed-key-switch | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Refresh after a model switch (`F12.managed-key-switch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-28-refresh-after-a-model-switch-f12-managed.json | +| F12.runtime-cold | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Runtime of an unloaded conversation (`F12.runtime-cold`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-07-runtime-of-an-unloaded-conversation-f12-.json | +| F12.runtime-errors | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Runtime errors (`F12.runtime-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-04-runtime-errors-f12-runtime-errors.json | +| F12.runtime-info | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Runtime info (`F12.runtime-info`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-02-runtime-info-f12-runtime-info.json | +| F12.runtime-launched-profile | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Launched profile from the runtime (`F12.runtime-launched-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-25-launched-profile-from-the-runtime-f12-ru.json | +| F12.runtime-reprovision | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Reprovision (`F12.runtime-reprovision`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-03-reprovision-f12-runtime-reprovision.json | +| F12.sandbox-pause-rehydrate | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Rehydrate after a pause (`F12.sandbox-pause-rehydrate`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-08-rehydrate-after-a-pause-f12-sandbox-paus.json | +| F12.sandbox-pause-releases | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Pause releases everything (`F12.sandbox-pause-releases`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-06-pause-releases-everything-f12-sandbox-pa.json | +| F12.sandbox-pause-running | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: Pause a running conversation (`F12.sandbox-pause-running`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-09-pause-a-running-conversation-f12-sandbox.json | +| F12.ts-client | 20261111-083146-0c07-map-f12 | beb261acfc6865e97384e53b3abdc6eb5c51a7a5 | pass | map run F12-runtime-and-credentials.md: TypeScript client (`F12.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083146-0c07-map-f12/evidence/map-run/F12-runtime-and-credentials-05-typescript-client-f12-ts-client.json | +| F13.auth | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Auth (`F13.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-01-auth-f13-auth.json | +| F13.clear | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Clear (`F13.clear`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-25-clear-f13-clear.json | +| F13.env-config-credentials | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Other credential variables (`F13.env-config-credentials`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-05-other-credential-variables-f13-env-confi.json | +| F13.env-sanitized | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Sanitized environment (`F13.env-sanitized`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-04-sanitized-environment-f13-env-sanitized.json | +| F13.execute-large-output | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Large output through execute (`F13.execute-large-output`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-16-large-output-through-execute-f13-execute.json | +| F13.execute-sync | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Synchronous execute (`F13.execute-sync`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-02-synchronous-execute-f13-execute-sync.json | +| F13.get-events | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Single and batch reads (`F13.get-events`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-11-single-and-batch-reads-f13-get-events.json | +| F13.global-cwd | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Global cwd (`F13.global-cwd`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-03-global-cwd-f13-global-cwd.json | +| F13.order-gt-paging | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Order filter and paging (`F13.order-gt-paging`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-15-order-filter-and-paging-f13-order-gt-pag.json | +| F13.output-chunks | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Output chunks (`F13.output-chunks`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-14-output-chunks-f13-output-chunks.json | +| F13.persist-restart | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Restart (`F13.persist-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-24-restart-f13-persist-restart.json | +| F13.retention | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Retention (`F13.retention`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-26-retention-f13-retention.json | +| F13.retention-rolling | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Rolling retention (`F13.retention-rolling`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-27-rolling-retention-f13-retention-rolling.json | +| F13.scoped-bad-id | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Malformed conversation id (`F13.scoped-bad-id`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-28-malformed-conversation-id-f13-scoped-bad.json | +| F13.scoped-cwd | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Scoped cwd (`F13.scoped-cwd`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-20-scoped-cwd-f13-scoped-cwd.json | +| F13.scoped-isolation | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Scope isolation (`F13.scoped-isolation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-22-scope-isolation-f13-scoped-isolation.json | +| F13.scoped-routes | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Scoped routes (`F13.scoped-routes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-21-scoped-routes-f13-scoped-routes.json | +| F13.scoped-unknown | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Unknown conversation (`F13.scoped-unknown`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-23-unknown-conversation-f13-scoped-unknown.json | +| F13.search | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Search (`F13.search`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-12-search-f13-search.json | +| F13.start-async | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Asynchronous start (`F13.start-async`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-06-asynchronous-start-f13-start-async.json | +| F13.stop | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Stop one command (`F13.stop`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-18-stop-one-command-f13-stop.json | +| F13.stop-idempotent | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Stop is idempotent (`F13.stop-idempotent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-19-stop-is-idempotent-f13-stop-idempotent.json | +| F13.timeout | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Timeout (`F13.timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-17-timeout-f13-timeout.json | +| F13.timestamp-offset | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Offset timestamps (`F13.timestamp-offset`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-13-offset-timestamps-f13-timestamp-offset.json | +| F13.ws-command | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Commands over the socket (`F13.ws-command`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-08-commands-over-the-socket-f13-ws-command.json | +| F13.ws-live | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Live and replayed socket frames (`F13.ws-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-07-live-and-replayed-socket-frames-f13-ws-l.json | +| F13.ws-resend-all | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Deprecated replay flag (`F13.ws-resend-all`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-09-deprecated-replay-flag-f13-ws-resend-all.json | +| F13.ws-subscriber-cap | 20261111-083201-c951-map-f13 | 3d2907868cde87fa84a9820ac4bebee7ff04f9b5 | pass | map run F13-bash.md: Subscriber cap (`F13.ws-subscriber-cap`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083201-c951-map-f13/evidence/map-run/F13-bash-10-subscriber-cap-f13-ws-subscriber-cap.json | +| F14.archive-auto-base | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Automatic base with a remote (`F14.archive-auto-base`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-17-automatic-base-with-a-remote-f14-archive.json | +| F14.archive-errors | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Archive validation (`F14.archive-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-20-archive-validation-f14-archive-errors.json | +| F14.archive-excludes | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Exclude controls (`F14.archive-excludes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-15-exclude-controls-f14-archive-excludes.json | +| F14.archive-format-doc | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: What `format` promises (`F14.archive-format-doc`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-19-what-format-promises-f14-archive-format-.json | +| F14.archive-git-delta | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: git-delta patch (`F14.archive-git-delta`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-16-git-delta-patch-f14-archive-git-delta.json | +| F14.archive-nested-repo | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Nested repository (`F14.archive-nested-repo`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-18-nested-repository-f14-archive-nested-rep.json | +| F14.archive-tar | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: tar.gz archive (`F14.archive-tar`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-14-tar-gz-archive-f14-archive-tar.json | +| F14.auth | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Session key required (`F14.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-01-session-key-required-f14-auth.json | +| F14.download-errors | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Download errors (`F14.download-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-07-download-errors-f14-download-errors.json | +| F14.home | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Home and favorites (`F14.home`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-09-home-and-favorites-f14-home.json | +| F14.mkdir | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Create directories (`F14.mkdir`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-08-create-directories-f14-mkdir.json | +| F14.restart | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Restart (`F14.restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-29-restart-f14-restart.json | +| F14.scoped-conversation-errors | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Unknown or malformed conversation (`F14.scoped-conversation-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-26-unknown-or-malformed-conversation-f14-sc.json | +| F14.scoped-path-guard | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Workspace path guard (`F14.scoped-path-guard`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-25-workspace-path-guard-f14-scoped-path-gua.json | +| F14.scoped-trajectory-archive | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Scoped trajectory and archive (`F14.scoped-trajectory-archive`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-27-scoped-trajectory-and-archive-f14-scoped.json | +| F14.scoped-transfer | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Conversation-scoped transfer (`F14.scoped-transfer`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-24-conversation-scoped-transfer-f14-scoped-.json | +| F14.sdk-workspace | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: SDK workspace (`F14.sdk-workspace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-28-sdk-workspace-f14-sdk-workspace.json | +| F14.subdirs | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: List subdirectories (`F14.subdirs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-10-list-subdirectories-f14-subdirs.json | +| F14.subdirs-case-pagination | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Paging past a case-only twin (`F14.subdirs-case-pagination`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-13-paging-past-a-case-only-twin-f14-subdirs.json | +| F14.subdirs-errors | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Listing errors (`F14.subdirs-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-12-listing-errors-f14-subdirs-errors.json | +| F14.subdirs-pagination | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Pagination (`F14.subdirs-pagination`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-11-pagination-f14-subdirs-pagination.json | +| F14.trajectory | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Trajectory zip (`F14.trajectory`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-21-trajectory-zip-f14-trajectory.json | +| F14.trajectory-concurrent | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Concurrent trajectory downloads (`F14.trajectory-concurrent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-23-concurrent-trajectory-downloads-f14-traj.json | +| F14.trajectory-errors | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Trajectory errors (`F14.trajectory-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-22-trajectory-errors-f14-trajectory-errors.json | +| F14.trajectory-init-path | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Init-time conversations path (`F14.trajectory-init-path`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-30-init-time-conversations-path-f14-traject.json | +| F14.upload-download | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Upload and download round trip (`F14.upload-download`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-02-upload-and-download-round-trip-f14-uploa.json | +| F14.upload-errors | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Upload validation (`F14.upload-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-03-upload-validation-f14-upload-errors.json | +| F14.upload-onto-directory | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Upload onto a directory (`F14.upload-onto-directory`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-04-upload-onto-a-directory-f14-upload-onto-.json | +| F14.upload-size | 20261111-083306-2a5a-map-f14 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F14-files.md: Upload size in the consumers' result (`F14.upload-size`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083306-2a5a-map-f14/evidence/map-run/F14-files-05-upload-size-in-the-consumers-result-f14-.json | +| F15.agent-changes | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: What the agent changed (`F15.agent-changes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-23-what-the-agent-changed-f15-agent-changes.json | +| F15.agent-changes | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: What the agent changed (`F15.agent-changes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-23-what-the-agent-changed-f15-agent-changes.json | +| F15.agent-changes | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: What the agent changed (`F15.agent-changes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-23-what-the-agent-changed-f15-agent-changes.json | +| F15.auth | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Session key required (`F15.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-01-session-key-required-f15-auth.json | +| F15.auth | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Session key required (`F15.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-01-session-key-required-f15-auth.json | +| F15.auth | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Session key required (`F15.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-01-session-key-required-f15-auth.json | +| F15.changes | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Working-tree changes (`F15.changes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-02-working-tree-changes-f15-changes.json | +| F15.changes | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Working-tree changes (`F15.changes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-02-working-tree-changes-f15-changes.json | +| F15.changes | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Working-tree changes (`F15.changes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-02-working-tree-changes-f15-changes.json | +| F15.changes-base | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Default display base (`F15.changes-base`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-04-default-display-base-f15-changes-base.json | +| F15.changes-base | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Default display base (`F15.changes-base`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-04-default-display-base-f15-changes-base.json | +| F15.changes-base | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Default display base (`F15.changes-base`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-04-default-display-base-f15-changes-base.json | +| F15.changes-nested-repo | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Nested repository (`F15.changes-nested-repo`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-06-nested-repository-f15-changes-nested-rep.json | +| F15.changes-nested-repo | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Nested repository (`F15.changes-nested-repo`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-06-nested-repository-f15-changes-nested-rep.json | +| F15.changes-nested-repo | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Nested repository (`F15.changes-nested-repo`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-06-nested-repository-f15-changes-nested-rep.json | +| F15.changes-ref | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Explicit base and its errors (`F15.changes-ref`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-05-explicit-base-and-its-errors-f15-changes.json | +| F15.changes-ref | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Explicit base and its errors (`F15.changes-ref`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-05-explicit-base-and-its-errors-f15-changes.json | +| F15.changes-ref | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Explicit base and its errors (`F15.changes-ref`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-05-explicit-base-and-its-errors-f15-changes.json | +| F15.changes-repo-below-path | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Repository below the requested directory (`F15.changes-repo-below-path`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-07-repository-below-the-requested-directory.json | +| F15.changes-repo-below-path | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Repository below the requested directory (`F15.changes-repo-below-path`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-07-repository-below-the-requested-directory.json | +| F15.changes-repo-below-path | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Repository below the requested directory (`F15.changes-repo-below-path`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-07-repository-below-the-requested-directory.json | +| F15.changes-special-names | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: File names with a space or non-ASCII characters (`F15.changes-special-names`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-08-file-names-with-a-space-or-non-ascii-cha.json | +| F15.changes-special-names | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: File names with a space or non-ASCII characters (`F15.changes-special-names`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-08-file-names-with-a-space-or-non-ascii-cha.json | +| F15.changes-special-names | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: File names with a space or non-ASCII characters (`F15.changes-special-names`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-08-file-names-with-a-space-or-non-ascii-cha.json | +| F15.changes-statuses | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Staged deletions, additions and renames (`F15.changes-statuses`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-03-staged-deletions-additions-and-renames-f.json | +| F15.changes-statuses | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Staged deletions, additions and renames (`F15.changes-statuses`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-03-staged-deletions-additions-and-renames-f.json | +| F15.changes-statuses | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Staged deletions, additions and renames (`F15.changes-statuses`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-03-staged-deletions-additions-and-renames-f.json | +| F15.changes-subdir-paths | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Subdirectory of a repository (`F15.changes-subdir-paths`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-09-subdirectory-of-a-repository-f15-changes.json | +| F15.changes-subdir-paths | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Subdirectory of a repository (`F15.changes-subdir-paths`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-09-subdirectory-of-a-repository-f15-changes.json | +| F15.changes-subdir-paths | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Subdirectory of a repository (`F15.changes-subdir-paths`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-09-subdirectory-of-a-repository-f15-changes.json | +| F15.commit-changes | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Files of one commit (`F15.commit-changes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-17-files-of-one-commit-f15-commit-changes.json | +| F15.commit-changes | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Files of one commit (`F15.commit-changes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-17-files-of-one-commit-f15-commit-changes.json | +| F15.commit-changes | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Files of one commit (`F15.commit-changes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-17-files-of-one-commit-f15-commit-changes.json | +| F15.commits | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Commit history (`F15.commits`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-16-commit-history-f15-commits.json | +| F15.commits | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Commit history (`F15.commits`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-16-commit-history-f15-commits.json | +| F15.commits | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Commit history (`F15.commits`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-16-commit-history-f15-commits.json | +| F15.diff | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Per-file diff (`F15.diff`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-11-per-file-diff-f15-diff.json | +| F15.diff | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Per-file diff (`F15.diff`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-11-per-file-diff-f15-diff.json | +| F15.diff | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Per-file diff (`F15.diff`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-11-per-file-diff-f15-diff.json | +| F15.diff-commit | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Diff of one commit (`F15.diff-commit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-12-diff-of-one-commit-f15-diff-commit.json | +| F15.diff-commit | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Diff of one commit (`F15.diff-commit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-12-diff-of-one-commit-f15-diff-commit.json | +| F15.diff-commit | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Diff of one commit (`F15.diff-commit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-12-diff-of-one-commit-f15-diff-commit.json | +| F15.diff-deleted-file | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Diff of a deleted file (`F15.diff-deleted-file`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-15-diff-of-a-deleted-file-f15-diff-deleted-.json | +| F15.diff-deleted-file | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Diff of a deleted file (`F15.diff-deleted-file`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-15-diff-of-a-deleted-file-f15-diff-deleted-.json | +| F15.diff-deleted-file | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Diff of a deleted file (`F15.diff-deleted-file`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-15-diff-of-a-deleted-file-f15-diff-deleted-.json | +| F15.diff-errors | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Diff errors (`F15.diff-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-13-diff-errors-f15-diff-errors.json | +| F15.diff-errors | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Diff errors (`F15.diff-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-13-diff-errors-f15-diff-errors.json | +| F15.diff-errors | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Diff errors (`F15.diff-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-13-diff-errors-f15-diff-errors.json | +| F15.diff-whitespace | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Unchanged-file whitespace (`F15.diff-whitespace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-14-unchanged-file-whitespace-f15-diff-white.json | +| F15.diff-whitespace | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Unchanged-file whitespace (`F15.diff-whitespace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-14-unchanged-file-whitespace-f15-diff-white.json | +| F15.diff-whitespace | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Unchanged-file whitespace (`F15.diff-whitespace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-14-unchanged-file-whitespace-f15-diff-white.json | +| F15.non-repo | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Not a repository, or no commits yet (`F15.non-repo`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-10-not-a-repository-or-no-commits-yet-f15-n.json | +| F15.non-repo | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Not a repository, or no commits yet (`F15.non-repo`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-10-not-a-repository-or-no-commits-yet-f15-n.json | +| F15.non-repo | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Not a repository, or no commits yet (`F15.non-repo`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-10-not-a-repository-or-no-commits-yet-f15-n.json | +| F15.pr-status-errors | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Pull-request status validation (`F15.pr-status-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-31-pull-request-status-validation-f15-pr-st.json | +| F15.pr-status-errors | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Pull-request status validation (`F15.pr-status-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-31-pull-request-status-validation-f15-pr-st.json | +| F15.pr-status-errors | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Pull-request status validation (`F15.pr-status-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-31-pull-request-status-validation-f15-pr-st.json | +| F15.pr-status-provider-errors | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Provider errors for pull-request status (`F15.pr-status-provider-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-34-provider-errors-for-pull-request-status-.json | +| F15.pr-status-provider-errors | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Provider errors for pull-request status (`F15.pr-status-provider-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-34-provider-errors-for-pull-request-status-.json | +| F15.pr-status-provider-errors | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Provider errors for pull-request status (`F15.pr-status-provider-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-34-provider-errors-for-pull-request-status-.json | +| F15.pr-status-state | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Pull-request state and CI result (`F15.pr-status-state`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-33-pull-request-state-and-ci-result-f15-pr-.json | +| F15.pr-status-state | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Pull-request state and CI result (`F15.pr-status-state`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-33-pull-request-state-and-ci-result-f15-pr-.json | +| F15.pr-status-state | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Pull-request state and CI result (`F15.pr-status-state`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-33-pull-request-state-and-ci-result-f15-pr-.json | +| F15.pr-status-timeout | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Pull-request status timeout (`F15.pr-status-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-35-pull-request-status-timeout-f15-pr-statu.json | +| F15.pr-status-timeout | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Pull-request status timeout (`F15.pr-status-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-35-pull-request-status-timeout-f15-pr-statu.json | +| F15.pr-status-timeout | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Pull-request status timeout (`F15.pr-status-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-35-pull-request-status-timeout-f15-pr-statu.json | +| F15.pr-status-token | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Pull-request status without a token (`F15.pr-status-token`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-30-pull-request-status-without-a-token-f15-.json | +| F15.pr-status-token | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Pull-request status without a token (`F15.pr-status-token`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-30-pull-request-status-without-a-token-f15-.json | +| F15.pr-status-token | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Pull-request status without a token (`F15.pr-status-token`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-30-pull-request-status-without-a-token-f15-.json | +| F15.pr-status-unreachable | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: GitHub unreachable for pull-request status (`F15.pr-status-unreachable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-32-github-unreachable-for-pull-request-stat.json | +| F15.pr-status-unreachable | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: GitHub unreachable for pull-request status (`F15.pr-status-unreachable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-32-github-unreachable-for-pull-request-stat.json | +| F15.pr-status-unreachable | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: GitHub unreachable for pull-request status (`F15.pr-status-unreachable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-32-github-unreachable-for-pull-request-stat.json | +| F15.repo-search-errors | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Provider and parameter validation (`F15.repo-search-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-25-provider-and-parameter-validation-f15-re.json | +| F15.repo-search-errors | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Provider and parameter validation (`F15.repo-search-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-25-provider-and-parameter-validation-f15-re.json | +| F15.repo-search-errors | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Provider and parameter validation (`F15.repo-search-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-25-provider-and-parameter-validation-f15-re.json | +| F15.repo-search-rejected-token | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Rejected provider token (`F15.repo-search-rejected-token`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-29-rejected-provider-token-f15-repo-search-.json | +| F15.repo-search-rejected-token | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Rejected provider token (`F15.repo-search-rejected-token`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-29-rejected-provider-token-f15-repo-search-.json | +| F15.repo-search-rejected-token | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Rejected provider token (`F15.repo-search-rejected-token`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-29-rejected-provider-token-f15-repo-search-.json | +| F15.repo-search-timeout | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: GitHub timeout (`F15.repo-search-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-27-github-timeout-f15-repo-search-timeout.json | +| F15.repo-search-timeout | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: GitHub timeout (`F15.repo-search-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-27-github-timeout-f15-repo-search-timeout.json | +| F15.repo-search-timeout | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: GitHub timeout (`F15.repo-search-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-27-github-timeout-f15-repo-search-timeout.json | +| F15.repo-search-token | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Token lookup (`F15.repo-search-token`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-24-token-lookup-f15-repo-search-token.json | +| F15.repo-search-token | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Token lookup (`F15.repo-search-token`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-24-token-lookup-f15-repo-search-token.json | +| F15.repo-search-token | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Token lookup (`F15.repo-search-token`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-24-token-lookup-f15-repo-search-token.json | +| F15.repo-search-unreachable | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: GitHub unreachable (`F15.repo-search-unreachable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-26-github-unreachable-f15-repo-search-unrea.json | +| F15.repo-search-unreachable | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: GitHub unreachable (`F15.repo-search-unreachable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-26-github-unreachable-f15-repo-search-unrea.json | +| F15.repo-search-unreachable | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: GitHub unreachable (`F15.repo-search-unreachable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-26-github-unreachable-f15-repo-search-unrea.json | +| F15.scoped | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Conversation-scoped views (`F15.scoped`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-18-conversation-scoped-views-f15-scoped.json | +| F15.scoped | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Conversation-scoped views (`F15.scoped`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-18-conversation-scoped-views-f15-scoped.json | +| F15.scoped | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Conversation-scoped views (`F15.scoped`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-18-conversation-scoped-views-f15-scoped.json | +| F15.scoped-guard | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Workspace guard (`F15.scoped-guard`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-19-workspace-guard-f15-scoped-guard.json | +| F15.scoped-guard | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Workspace guard (`F15.scoped-guard`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-19-workspace-guard-f15-scoped-guard.json | +| F15.scoped-guard | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Workspace guard (`F15.scoped-guard`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-19-workspace-guard-f15-scoped-guard.json | +| F15.scoped-restart | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: After a restart (`F15.scoped-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-20-after-a-restart-f15-scoped-restart.json | +| F15.scoped-restart | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: After a restart (`F15.scoped-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-20-after-a-restart-f15-scoped-restart.json | +| F15.scoped-restart | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: After a restart (`F15.scoped-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-20-after-a-restart-f15-scoped-restart.json | +| F15.sdk-workspace | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Python SDK workspace (`F15.sdk-workspace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-21-python-sdk-workspace-f15-sdk-workspace.json | +| F15.sdk-workspace | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: Python SDK workspace (`F15.sdk-workspace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-21-python-sdk-workspace-f15-sdk-workspace.json | +| F15.sdk-workspace | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: Python SDK workspace (`F15.sdk-workspace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-21-python-sdk-workspace-f15-sdk-workspace.json | +| F15.ts-client | 20261111-084346-c245-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: TypeScript client (`F15.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084346-c245-map-f15/evidence/map-run/F15-git-22-typescript-client-f15-ts-client.json | +| F15.ts-client | 20261111-084831-3c8b-map-f15 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F15-git.md: TypeScript client (`F15.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-3c8b-map-f15/evidence/map-run/F15-git-22-typescript-client-f15-ts-client.json | +| F15.ts-client | 20261111-085423-8ec2-map-f15 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F15-git.md: TypeScript client (`F15.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085423-8ec2-map-f15/evidence/map-run/F15-git-22-typescript-client-f15-ts-client.json | +| F16.agent-artifact | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: The agent's own artifact (`F16.agent-artifact`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-14-the-agent-s-own-artifact-f16-agent-artif.json | +| F16.auth | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Header or cookie (`F16.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-13-header-or-cookie-f16-auth.json | +| F16.content-types | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Content types (`F16.content-types`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-03-content-types-f16-content-types.json | +| F16.deleted-conversation | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Deleted conversation (`F16.deleted-conversation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-11-deleted-conversation-f16-deleted-convers.json | +| F16.dir-index | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Directory index and missing files (`F16.dir-index`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-04-directory-index-and-missing-files-f16-di.json | +| F16.file-bytes | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: File bytes and headers (`F16.file-bytes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-02-file-bytes-and-headers-f16-file-bytes.json | +| F16.live-content | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Live content (`F16.live-content`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-06-live-content-f16-live-content.json | +| F16.range | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Byte ranges (`F16.range`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-05-byte-ranges-f16-range.json | +| F16.restart | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Across a restart (`F16.restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-15-across-a-restart-f16-restart.json | +| F16.root-index | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Root index (`F16.root-index`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-01-root-index-f16-root-index.json | +| F16.symlink-escape | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Symlinks (`F16.symlink-escape`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-08-symlinks-f16-symlink-escape.json | +| F16.traversal-rejected | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Traversal (`F16.traversal-rejected`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-07-traversal-f16-traversal-rejected.json | +| F16.unknown-conversation | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Unknown and malformed ids (`F16.unknown-conversation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-09-unknown-and-malformed-ids-f16-unknown-co.json | +| F16.vscode-auth | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: VS Code routes are header-only (`F16.vscode-auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-18-vs-code-routes-are-header-only-f16-vscod.json | +| F16.vscode-disabled | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: VS Code disabled (`F16.vscode-disabled`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-16-vs-code-disabled-f16-vscode-disabled.json | +| F16.vscode-enabled-status | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Enabled but not running (`F16.vscode-enabled-status`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-19-enabled-but-not-running-f16-vscode-enabl.json | +| F16.vscode-scoped-guards | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Scoped guards (`F16.vscode-scoped-guards`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-17-scoped-guards-f16-vscode-scoped-guards.json | +| F16.vscode-url-after-init | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Key delivered by init, no URL (`F16.vscode-url-after-init`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-24-key-delivered-by-init-no-url-f16-vscode-.json | +| F16.vscode-url-no-key | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: No key, no URL (`F16.vscode-url-no-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-23-no-key-no-url-f16-vscode-url-no-key.json | +| F16.vscode-url-not-running | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Not running, not advertised (`F16.vscode-url-not-running`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-21-not-running-not-advertised-f16-vscode-ur.json | +| F16.workspace-dir-missing | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Workspace directory gone (`F16.workspace-dir-missing`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-10-workspace-directory-gone-f16-workspace-d.json | +| F16.worktree-served | 20261111-085638-3cbc-map-f16 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F16-workspace-serving-and-vscode.md: Worktree conversation (`F16.worktree-served`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085638-3cbc-map-f16/evidence/map-run/F16-workspace-serving-and-vscode-12-worktree-conversation-f16-worktree-serve.json | +| F17.add-parents | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Register parents (`F17.add-parents`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-11-register-parents-f17-add-parents.json | +| F17.add-workspaces | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Register a repository and a plain directory (`F17.add-workspaces`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-03-register-a-repository-and-a-plain-direct.json | +| F17.auth-required | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Session key required (`F17.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-16-session-key-required-f17-auth-required.json | +| F17.concurrent-adds | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Concurrent writes (`F17.concurrent-adds`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-17-concurrent-writes-f17-concurrent-adds.json | +| F17.corrupted-file | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Corrupted file (`F17.corrupted-file`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-20-corrupted-file-f17-corrupted-file.json | +| F17.delete-not-found | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Unknown paths (`F17.delete-not-found`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-07-unknown-paths-f17-delete-not-found.json | +| F17.delete-parent | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Delete a parent (`F17.delete-parent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-13-delete-a-parent-f17-delete-parent.json | +| F17.delete-workspace | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Delete a workspace (`F17.delete-workspace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-06-delete-a-workspace-f17-delete-workspace.json | +| F17.future-schema | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Future or invalid schema (`F17.future-schema`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-21-future-or-invalid-schema-f17-future-sche.json | +| F17.idempotent-add | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Idempotent re-add (`F17.idempotent-add`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-05-idempotent-re-add-f17-idempotent-add.json | +| F17.io-error | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Unreadable file (`F17.io-error`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-22-unreadable-file-f17-io-error.json | +| F17.list-empty | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Empty registry (`F17.list-empty`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-01-empty-registry-f17-list-empty.json | +| F17.lists-independent | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Two independent lists (`F17.lists-independent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-14-two-independent-lists-f17-lists-independ.json | +| F17.noop-delete-no-file | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: A refused delete on a fresh directory (`F17.noop-delete-no-file`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-02-a-refused-delete-on-a-fresh-directory-f1.json | +| F17.noop-write-readonly | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: No-op writes on read-only storage (`F17.noop-write-readonly`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-23-no-op-writes-on-read-only-storage-f17-no.json | +| F17.noop-write-untouched | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: No-op writes leave the file alone (`F17.noop-write-untouched`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-08-no-op-writes-leave-the-file-alone-f17-no.json | +| F17.parent-children | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Children of a parent (`F17.parent-children`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-12-children-of-a-parent-f17-parent-children.json | +| F17.paths-opaque | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Paths are opaque strings (`F17.paths-opaque`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-09-paths-are-opaque-strings-f17-paths-opaqu.json | +| F17.persist-restart | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Persistence across a restart (`F17.persist-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-15-persistence-across-a-restart-f17-persist.json | +| F17.persistence-location | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Location without `OH_PERSISTENCE_DIR` (`F17.persistence-location`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-24-location-without-oh-persistence-dir-f17-.json | +| F17.shared-dir | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Two servers, one file (`F17.shared-dir`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-18-two-servers-one-file-f17-shared-dir.json | +| F17.ts-client | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: TypeScript client (`F17.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-19-typescript-client-f17-ts-client.json | +| F17.validation | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Validation (`F17.validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-10-validation-f17-validation.json | +| F17.wire-format | 20261111-085659-a02a-map-f17 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F17-workspaces-registry.md: Wire format (`F17.wire-format`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085659-a02a-map-f17/evidence/map-run/F17-workspaces-registry-04-wire-format-f17-wire-format.json | +| F18.active-profile-pointer | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Active LLM profile pointer (`F18.active-profile-pointer`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-12-active-llm-profile-pointer-f18-active-pr.json | +| F18.agent-diff-merge | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Sparse agent diff and unset (`F18.agent-diff-merge`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-08-sparse-agent-diff-and-unset-f18-agent-di.json | +| F18.agent-kind-switch | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Agent kind switch (`F18.agent-kind-switch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-13-agent-kind-switch-f18-agent-kind-switch.json | +| F18.agent-schema | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Agent settings schema (`F18.agent-schema`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-02-agent-settings-schema-f18-agent-schema.json | +| F18.auth-required | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Key required (`F18.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-04-key-required-f18-auth-required.json | +| F18.conversation-diff | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Conversation settings (`F18.conversation-diff`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-09-conversation-settings-f18-conversation-d.json | +| F18.conversation-schema | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Conversation settings schema (`F18.conversation-schema`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-03-conversation-settings-schema-f18-convers.json | +| F18.corrupted-files | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Corrupted files (`F18.corrupted-files`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-23-corrupted-files-f18-corrupted-files.json | +| F18.dangling-pointers | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Dangling meta-profile pointer (`F18.dangling-pointers`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-28-dangling-meta-profile-pointer-f18-dangli.json | +| F18.encrypted-at-rest | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Encrypted at rest (`F18.encrypted-at-rest`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-17-encrypted-at-rest-f18-encrypted-at-rest.json | +| F18.expose-modes | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Exposure modes (`F18.expose-modes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-06-exposure-modes-f18-expose-modes.json | +| F18.get-defaults | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Fresh defaults (`F18.get-defaults`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-01-fresh-defaults-f18-get-defaults.json | +| F18.misc-settings-merge | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Opaque misc settings (`F18.misc-settings-merge`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-10-opaque-misc-settings-f18-misc-settings-m.json | +| F18.no-cipher | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: No cipher (`F18.no-cipher`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-24-no-cipher-f18-no-cipher.json | +| F18.patch-llm | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Set the LLM (`F18.patch-llm`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-05-set-the-llm-f18-patch-llm.json | +| F18.patch-validation | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Rejected PATCH bodies (`F18.patch-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-11-rejected-patch-bodies-f18-patch-validati.json | +| F18.persist-restart | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Persistence across a restart (`F18.persist-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-22-persistence-across-a-restart-f18-persist.json | +| F18.roundtrip | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Post back what GET returned (`F18.roundtrip`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-07-post-back-what-get-returned-f18-roundtri.json | +| F18.sdk-clone-repos | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: SDK clones with the stored token (`F18.sdk-clone-repos`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-20-sdk-clones-with-the-stored-token-f18-sdk.json | +| F18.sdk-remote-workspace | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: SDK consumer (`F18.sdk-remote-workspace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-19-sdk-consumer-f18-sdk-remote-workspace.json | +| F18.secret-delete | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Delete a secret (`F18.secret-delete`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-15-delete-a-secret-f18-secret-delete.json | +| F18.secret-empty-value | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Empty secret value (`F18.secret-empty-value`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-26-empty-secret-value-f18-secret-empty-valu.json | +| F18.secret-key-previous-keys | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Supported key rotation (`F18.secret-key-previous-keys`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-33-supported-key-rotation-f18-secret-key-pr.json | +| F18.secret-key-rotation | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Cipher key lost (`F18.secret-key-rotation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-32-cipher-key-lost-f18-secret-key-rotation.json | +| F18.secret-name-contract | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Documented status for an invalid name (`F18.secret-name-contract`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-31-documented-status-for-an-invalid-name-f1.json | +| F18.secret-name-validation | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Secret names (`F18.secret-name-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-16-secret-names-f18-secret-name-validation.json | +| F18.secret-put-list-get | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Upsert, list and read a secret (`F18.secret-put-list-get`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-14-upsert-list-and-read-a-secret-f18-secret.json | +| F18.secret-scope-agent-profile | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Secrets scoped by an agent profile (`F18.secret-scope-agent-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-18-secrets-scoped-by-an-agent-profile-f18-s.json | +| F18.secret-scope-profile-errors | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: Agent-profile store busy (`F18.secret-scope-profile-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-30-agent-profile-store-busy-f18-secret-scop.json | +| F18.ts-client | 20261111-083602-db6e-map-f18 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F18-settings-and-secrets.md: TypeScript client (`F18.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083602-db6e-map-f18/evidence/map-run/F18-settings-and-secrets-21-typescript-client-f18-ts-client.json | +| F19.auth | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Auth (`F19.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-01-auth-f19-auth.json | +| F19.conversation-secrets-at-rest | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Credentials at rest (`F19.conversation-secrets-at-rest`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-27-credentials-at-rest-f19-conversation-sec.json | +| F19.conversation-tool | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: The agent uses the MCP tool (`F19.conversation-tool`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-25-the-agent-uses-the-mcp-tool-f19-conversa.json | +| F19.disabled-server | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Disabled servers stay off (`F19.disabled-server`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-24-disabled-servers-stay-off-f19-disabled-s.json | +| F19.oauth-callback-bad-port | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Port out of range (`F19.oauth-callback-bad-port`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-18-port-out-of-range-f19-oauth-callback-bad.json | +| F19.oauth-callback-replay | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Replayed callback (`F19.oauth-callback-replay`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-17-replayed-callback-f19-oauth-callback-rep.json | +| F19.oauth-callback-validation | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Callback URL checks on a live job (`F19.oauth-callback-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-19-callback-url-checks-on-a-live-job-f19-oa.json | +| F19.oauth-consent-denied | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Denied consent (`F19.oauth-consent-denied`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-20-denied-consent-f19-oauth-consent-denied.json | +| F19.oauth-consent-timeout | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Consent timeout (`F19.oauth-consent-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-21-consent-timeout-f19-oauth-consent-timeou.json | +| F19.oauth-flow | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Full OAuth consent (`F19.oauth-flow`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-16-full-oauth-consent-f19-oauth-flow.json | +| F19.oauth-inline-writeback | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Inline OAuth server refresh (`F19.oauth-inline-writeback`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-22-inline-oauth-server-refresh-f19-oauth-in.json | +| F19.oauth-refresh-writeback | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Stored OAuth token refresh (`F19.oauth-refresh-writeback`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-26-stored-oauth-token-refresh-f19-oauth-ref.json | +| F19.oauth-start-errors | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: OAuth start refusals and early outcomes (`F19.oauth-start-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-14-oauth-start-refusals-and-early-outcomes-.json | +| F19.oauth-stored-state | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Stored OAuth state (`F19.oauth-stored-state`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-23-stored-oauth-state-f19-oauth-stored-stat.json | +| F19.oauth-unknown-job | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Unknown jobs (`F19.oauth-unknown-job`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-15-unknown-jobs-f19-oauth-unknown-job.json | +| F19.restart | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Restart (`F19.restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-28-restart-f19-restart.json | +| F19.settings-create | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Create one server (`F19.settings-create`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-02-create-one-server-f19-settings-create.json | +| F19.settings-delete | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Delete one server (`F19.settings-delete`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-06-delete-one-server-f19-settings-delete.json | +| F19.settings-empty-server | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Server with neither command nor url (`F19.settings-empty-server`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-04-server-with-neither-command-nor-url-f19-.json | +| F19.settings-patch | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Sparse patch (`F19.settings-patch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-05-sparse-patch-f19-settings-patch.json | +| F19.settings-validation | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Invalid servers (`F19.settings-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-03-invalid-servers-f19-settings-validation.json | +| F19.test-connect-failures | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Connection failures (`F19.test-connect-failures`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-10-connection-failures-f19-test-connect-fai.json | +| F19.test-encrypted-env | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Encrypted env from settings (`F19.test-encrypted-env`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-12-encrypted-env-from-settings-f19-test-enc.json | +| F19.test-stdio | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Test a stdio server (`F19.test-stdio`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-08-test-a-stdio-server-f19-test-stdio.json | +| F19.test-timeout | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Handshake timeout (`F19.test-timeout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-11-handshake-timeout-f19-test-timeout.json | +| F19.test-tool-errors | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Tool failures stay in `tool_result` (`F19.test-tool-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-09-tool-failures-stay-in-tool-result-f19-te.json | +| F19.test-validation | 20261111-085751-f467-map-f19 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F19-mcp.md: Connection test body validation (`F19.test-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085751-f467-map-f19/evidence/map-run/F19-mcp-07-connection-test-body-validation-f19-test.json | +| F20.activate | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Activate (`F20.activate`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-21-activate-f20-activate.json | +| F20.activate-acp | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: ACP settings (`F20.activate-acp`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-23-acp-settings-f20-activate-acp.json | +| F20.activate-conversation | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Activation drives conversations (`F20.activate-conversation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-22-activation-drives-conversations-f20-acti.json | +| F20.activate-dangling-provider | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Dangling provider link (`F20.activate-dangling-provider`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-26-dangling-provider-link-f20-activate-dang.json | +| F20.activate-linked-provider | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Linked provider (`F20.activate-linked-provider`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-27-linked-provider-f20-activate-linked-prov.json | +| F20.auth-required | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Key required (`F20.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-01-key-required-f20-auth-required.json | +| F20.corrupt-files | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Corrupted files (`F20.corrupt-files`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-09-corrupted-files-f20-corrupt-files.json | +| F20.delete | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Delete (`F20.delete`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-34-delete-f20-delete.json | +| F20.delete-guarded | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Guarded delete (`F20.delete-guarded`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-33-guarded-delete-f20-delete-guarded.json | +| F20.get-secrets | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Reading the key (`F20.get-secrets`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-04-reading-the-key-f20-get-secrets.json | +| F20.json-alias-guard | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Delete through the `.json` alias (`F20.json-alias-guard`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-35-delete-through-the-json-alias-f20-json-a.json | +| F20.key-encrypted-at-rest | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Encrypted at rest (`F20.key-encrypted-at-rest`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-03-encrypted-at-rest-f20-key-encrypted-at-r.json | +| F20.name-rules | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Name rules (`F20.name-rules`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-08-name-rules-f20-name-rules.json | +| F20.no-cipher | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: No cipher (`F20.no-cipher`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-29-no-cipher-f20-no-cipher.json | +| F20.not-found | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Unknown names (`F20.not-found`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-07-unknown-names-f20-not-found.json | +| F20.persist-restart | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Restart (`F20.persist-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-30-restart-f20-persist-restart.json | +| F20.profile-limit | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Profile limit (`F20.profile-limit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-12-profile-limit-f20-profile-limit.json | +| F20.rename | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Rename (`F20.rename`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-31-rename-f20-rename.json | +| F20.rename-errors | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Rename errors (`F20.rename-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-32-rename-errors-f20-rename-errors.json | +| F20.save | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Save (`F20.save`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-02-save-f20-save.json | +| F20.save-encrypted-roundtrip | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Encrypted round trip (`F20.save-encrypted-roundtrip`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-05-encrypted-round-trip-f20-save-encrypted-.json | +| F20.save-without-secrets | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Save without secrets (`F20.save-without-secrets`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-06-save-without-secrets-f20-save-without-se.json | +| F20.sdk-get-llm | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: SDK consumer (`F20.sdk-get-llm`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-24-sdk-consumer-f20-sdk-get-llm.json | +| F20.sdk-get-llm-linked | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: SDK on a linked profile (`F20.sdk-get-llm-linked`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-28-sdk-on-a-linked-profile-f20-sdk-get-llm-.json | +| F20.secret-key-previous-keys | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Supported key rotation (`F20.secret-key-previous-keys`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-36-supported-key-rotation-f20-secret-key-pr.json | +| F20.secret-key-rotation | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Secret key lost (`F20.secret-key-rotation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-37-secret-key-lost-f20-secret-key-rotation.json | +| F20.store-busy | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Store busy (`F20.store-busy`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-10-store-busy-f20-store-busy.json | +| F20.store-busy-responsive | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Lock wait does not block the server (`F20.store-busy-responsive`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-11-lock-wait-does-not-block-the-server-f20-.json | +| F20.ts-client | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: TypeScript client (`F20.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-25-typescript-client-f20-ts-client.json | +| F20.validate-hang-bounded | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Hanging provider gets a bounded verdict (`F20.validate-hang-bounded`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-20-hanging-provider-gets-a-bounded-verdict-.json | +| F20.validate-live | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Validate a working config (`F20.validate-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-13-validate-a-working-config-f20-validate-l.json | +| F20.validate-provider-errors | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Provider failures (`F20.validate-provider-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-16-provider-failures-f20-validate-provider-.json | +| F20.validate-quota-exhausted | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Exhausted quota (`F20.validate-quota-exhausted`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-17-exhausted-quota-f20-validate-quota-exhau.json | +| F20.validate-rejects | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Validate rejects (`F20.validate-rejects`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-14-validate-rejects-f20-validate-rejects.json | +| F20.validate-subscription-login | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Subscription without a login (`F20.validate-subscription-login`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-18-subscription-without-a-login-f20-validat.json | +| F20.validate-unreachable-bounded | 20261111-085920-8f90-map-f20 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F20-llm-profiles.md: Unreachable endpoint answers at once (`F20.validate-unreachable-bounded`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085920-8f90-map-f20/evidence/map-run/F20-llm-profiles-19-unreachable-endpoint-answers-at-once-f20.json | +| F21.acp-kind | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: ACP profile (`F21.acp-kind`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-07-acp-profile-f21-acp-kind.json | +| F21.acp-skill-sourcing | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: ACP skill sourcing (`F21.acp-skill-sourcing`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-26-acp-skill-sourcing-f21-acp-skill-sourcin.json | +| F21.activate | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Activate by id (`F21.activate`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-11-activate-by-id-f21-activate.json | +| F21.activate-corrupt-settings | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Activate over a corrupt settings file (`F21.activate-corrupt-settings`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-12-activate-over-a-corrupt-settings-file-f2.json | +| F21.auth-required | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: No key, no access (`F21.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-01-no-key-no-access-f21-auth-required.json | +| F21.busy-blocks-server | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Lock wait does not stall the server (`F21.busy-blocks-server`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-29-lock-wait-does-not-stall-the-server-f21-.json | +| F21.conversation-start | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Start a conversation from a profile (`F21.conversation-start`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-21-start-a-conversation-from-a-profile-f21-.json | +| F21.corrupt-file | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Corrupt file (`F21.corrupt-file`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-20-corrupt-file-f21-corrupt-file.json | +| F21.dangling-llm-report | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: How a dangling LLM reference is reported (`F21.dangling-llm-report`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-30-how-a-dangling-llm-reference-is-reported.json | +| F21.delete | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Delete (`F21.delete`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-14-delete-f21-delete.json | +| F21.delete-active-acp-reset | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Active ACP reset (`F21.delete-active-acp-reset`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-15-active-acp-reset-f21-delete-active-acp-r.json | +| F21.json-suffix-identity | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: `.json` names keep their identity (`F21.json-suffix-identity`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-32-json-names-keep-their-identity-f21-json-.json | +| F21.launch-additions | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Launch additions (`F21.launch-additions`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-23-launch-additions-f21-launch-additions.json | +| F21.launch-store-busy | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Busy store at conversation start (`F21.launch-store-busy`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-28-busy-store-at-conversation-start-f21-lau.json | +| F21.lazy-seed | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Lazy seed (`F21.lazy-seed`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-02-lazy-seed-f21-lazy-seed.json | +| F21.limit | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: At most 50 (`F21.limit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-19-at-most-50-f21-limit.json | +| F21.materialize-dangling | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Dangling references (`F21.materialize-dangling`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-17-dangling-references-f21-materialize-dang.json | +| F21.materialize-draft | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Draft preview (`F21.materialize-draft`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-18-draft-preview-f21-materialize-draft.json | +| F21.materialize-valid | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Materialize a resolvable profile (`F21.materialize-valid`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-16-materialize-a-resolvable-profile-f21-mat.json | +| F21.name-rules | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Name rules (`F21.name-rules`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-09-name-rules-f21-name-rules.json | +| F21.not-found | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Unknown names and ids (`F21.not-found`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-10-unknown-names-and-ids-f21-not-found.json | +| F21.rename | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Rename (`F21.rename`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-13-rename-f21-rename.json | +| F21.restart | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Restart (`F21.restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-25-restart-f21-restart.json | +| F21.save-create | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Create (`F21.save-create`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-05-create-f21-save-create.json | +| F21.save-overwrite | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Overwrite (`F21.save-overwrite`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-06-overwrite-f21-save-overwrite.json | +| F21.save-validation | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Save validation (`F21.save-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-08-save-validation-f21-save-validation.json | +| F21.secret-scope | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Secret scope (`F21.secret-scope`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-22-secret-scope-f21-secret-scope.json | +| F21.seed-acp | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: ACP seed (`F21.seed-acp`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-04-acp-seed-f21-seed-acp.json | +| F21.seed-no-ghost-llm | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: No ghost LLM profile (`F21.seed-no-ghost-llm`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-03-no-ghost-llm-profile-f21-seed-no-ghost-l.json | +| F21.store-busy | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: Store busy (`F21.store-busy`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-27-store-busy-f21-store-busy.json | +| F21.ts-client | 20261111-083718-2007-map-f21 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F21-agent-profiles.md: TypeScript client (`F21.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083718-2007-map-f21/evidence/map-run/F21-agent-profiles-24-typescript-client-f21-ts-client.json | +| F22.acp-clears-pointer | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: ACP clears the pointer (`F22.acp-clears-pointer`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-16-acp-clears-the-pointer-f22-acp-clears-po.json | +| F22.activate | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Activate (`F22.activate`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-09-activate-f22-activate.json | +| F22.activate-acp | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Activation on ACP settings (`F22.activate-acp`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-17-activation-on-acp-settings-f22-activate-.json | +| F22.auth-required | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Key required (`F22.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-01-key-required-f22-auth-required.json | +| F22.corrupt-files | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Corrupted files (`F22.corrupt-files`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-18-corrupted-files-f22-corrupt-files.json | +| F22.corrupt-settings | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Unreadable settings (`F22.corrupt-settings`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-19-unreadable-settings-f22-corrupt-settings.json | +| F22.delete | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Delete (`F22.delete`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-14-delete-f22-delete.json | +| F22.delete-active-clears | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Delete the active one (`F22.delete-active-clears`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-15-delete-the-active-one-f22-delete-active-.json | +| F22.empty-router | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Router with no route (`F22.empty-router`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-05-router-with-no-route-f22-empty-router.json | +| F22.limit | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Limit (`F22.limit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-22-limit-f22-limit.json | +| F22.name-rules | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Name rules (`F22.name-rules`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-07-name-rules-f22-name-rules.json | +| F22.not-found | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Unknown names (`F22.not-found`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-08-unknown-names-f22-not-found.json | +| F22.overwrite | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Overwrite (`F22.overwrite`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-06-overwrite-f22-overwrite.json | +| F22.persist-restart | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Restart (`F22.persist-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-13-restart-f22-persist-restart.json | +| F22.persistence-dir | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Persistence directory with `~` (`F22.persistence-dir`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-24-persistence-directory-with-f22-persisten.json | +| F22.route-classes | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Classes-mode routing (`F22.route-classes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-10-classes-mode-routing-f22-route-classes.json | +| F22.route-direct | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Direct routing (`F22.route-direct`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-11-direct-routing-f22-route-direct.json | +| F22.route-missing-profile | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Missing LLM profile (`F22.route-missing-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-12-missing-llm-profile-f22-route-missing-pr.json | +| F22.save-classes | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Save in classes mode (`F22.save-classes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-02-save-in-classes-mode-f22-save-classes.json | +| F22.save-direct | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Save in direct mode (`F22.save-direct`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-03-save-in-direct-mode-f22-save-direct.json | +| F22.store-busy | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Store busy (`F22.store-busy`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-20-store-busy-f22-store-busy.json | +| F22.store-busy-responsive | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Lock wait does not block the server (`F22.store-busy-responsive`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-21-lock-wait-does-not-block-the-server-f22-.json | +| F22.ts-client | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: TypeScript client (`F22.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-23-typescript-client-f22-ts-client.json | +| F22.validation | 20261111-090015-c709-map-f22 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F22-meta-profiles.md: Validation (`F22.validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090015-c709-map-f22/evidence/map-run/F22-meta-profiles-04-validation-f22-validation.json | +| F23.acp-discovery-errors | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: ACP discovery errors (`F23.acp-discovery-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-27-acp-discovery-errors-f23-acp-discovery-e.json | +| F23.acp-discovery-live | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Real ACP model discovery (`F23.acp-discovery-live`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-28-real-acp-model-discovery-f23-acp-discove.json | +| F23.auth-required | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Key required (`F23.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-01-key-required-f23-auth-required.json | +| F23.catalog-ts-client | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: TypeScript client (`F23.catalog-ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-07-typescript-client-f23-catalog-ts-client.json | +| F23.conn-create | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Create (`F23.conn-create`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-08-create-f23-conn-create.json | +| F23.conn-create-blank-key | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Blank keys on create (`F23.conn-create-blank-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-12-blank-keys-on-create-f23-conn-create-bla.json | +| F23.conn-create-validation | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Create validation (`F23.conn-create-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-11-create-validation-f23-conn-create-valida.json | +| F23.conn-delete | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Delete (`F23.conn-delete`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-25-delete-f23-conn-delete.json | +| F23.conn-key-encrypted | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Key hidden and encrypted (`F23.conn-key-encrypted`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-09-key-hidden-and-encrypted-f23-conn-key-en.json | +| F23.conn-limit | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Limit (`F23.conn-limit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-26-limit-f23-conn-limit.json | +| F23.conn-patch-blank-key | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Blank keys on update (`F23.conn-patch-blank-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-16-blank-keys-on-update-f23-conn-patch-blan.json | +| F23.conn-rotate-key | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Rotate the key (`F23.conn-rotate-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-17-rotate-the-key-f23-conn-rotate-key.json | +| F23.conn-update | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Update (`F23.conn-update`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-14-update-f23-conn-update.json | +| F23.conn-update-validation | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Update validation (`F23.conn-update-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-15-update-validation-f23-conn-update-valida.json | +| F23.corrupt-file | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Corrupted file (`F23.corrupt-file`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-21-corrupted-file-f23-corrupt-file.json | +| F23.delete-guard | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Delete guard (`F23.delete-guard`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-24-delete-guard-f23-delete-guard.json | +| F23.linked-run | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Linked profile runs on the connection (`F23.linked-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-13-linked-profile-runs-on-the-connection-f2.json | +| F23.models | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: All models (`F23.models`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-03-all-models-f23-models.json | +| F23.models-filter | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Provider filter (`F23.models-filter`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-04-provider-filter-f23-models-filter.json | +| F23.models-no-bedrock | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: No Bedrock ids under a provider (`F23.models-no-bedrock`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-05-no-bedrock-ids-under-a-provider-f23-mode.json | +| F23.no-cipher | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: No cipher (`F23.no-cipher`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-10-no-cipher-f23-no-cipher.json | +| F23.persist-restart | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Restart (`F23.persist-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-19-restart-f23-persist-restart.json | +| F23.providers | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Providers (`F23.providers`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-02-providers-f23-providers.json | +| F23.store-busy | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Store busy (`F23.store-busy`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-22-store-busy-f23-store-busy.json | +| F23.store-busy-responsive | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Lock wait does not block the server (`F23.store-busy-responsive`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-23-lock-wait-does-not-block-the-server-f23-.json | +| F23.strict-decrypt | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Secret key changed (`F23.strict-decrypt`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-20-secret-key-changed-f23-strict-decrypt.json | +| F23.validate-linked-draft | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Validate a linked draft (`F23.validate-linked-draft`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-18-validate-a-linked-draft-f23-validate-lin.json | +| F23.verified-models | 20261111-083744-b49d-map-f23 | 07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72 | pass | map run F23-llm-catalog-and-connections.md: Verified models (`F23.verified-models`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-083744-b49d-map-f23/evidence/map-run/F23-llm-catalog-and-connections-06-verified-models-f23-verified-models.json | +| F24.auth-required | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Key required (`F24.auth-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-04-key-required-f24-auth-required.json | +| F24.device-poll-invalid | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Malformed polls (`F24.device-poll-invalid`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-09-malformed-polls-f24-device-poll-invalid.json | +| F24.device-poll-outage | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Outage during a pending login (`F24.device-poll-outage`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-17-outage-during-a-pending-login-f24-device.json | +| F24.device-poll-pending | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Pending login (`F24.device-poll-pending`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-12-pending-login-f24-device-poll-pending.json | +| F24.device-poll-unknown | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Unknown token (`F24.device-poll-unknown`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-10-unknown-token-f24-device-poll-unknown.json | +| F24.device-start | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Start a login (`F24.device-start`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-11-start-a-login-f24-device-start.json | +| F24.device-start-offline | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Start and poll a login offline (`F24.device-start-offline`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-19-start-and-poll-a-login-offline-f24-devic.json | +| F24.logout | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Logout (`F24.logout`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-06-logout-f24-logout.json | +| F24.logout-cancels-pending | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Logout cancels pending logins (`F24.logout-cancels-pending`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-13-logout-cancels-pending-logins-f24-logout.json | +| F24.models | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Subscription models (`F24.models`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-01-subscription-models-f24-models.json | +| F24.offline-safe | 20261111-084256-5c92-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Offline-safe routes (`F24.offline-safe`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084256-5c92-map-f24/evidence/map-run/F24-openai-subscription-01-offline-safe-routes-f24-offline-safe.json | +| F24.offline-safe | 20261111-084331-063f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Offline-safe routes (`F24.offline-safe`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084331-063f-map-f24/evidence/map-run/F24-openai-subscription-01-offline-safe-routes-f24-offline-safe.json | +| F24.offline-safe | 20261111-084410-5f54-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Offline-safe routes (`F24.offline-safe`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084410-5f54-map-f24/evidence/map-run/F24-openai-subscription-01-offline-safe-routes-f24-offline-safe.json | +| F24.pending-lost-on-restart | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Pending logins and restarts (`F24.pending-lost-on-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-14-pending-logins-and-restarts-f24-pending-.json | +| F24.status-connected | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Status with a stored login (`F24.status-connected`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-03-status-with-a-stored-login-f24-status-co.json | +| F24.status-disconnected | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Status with no login (`F24.status-disconnected`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-02-status-with-no-login-f24-status-disconne.json | +| F24.status-invalid-file | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Invalid credentials file (`F24.status-invalid-file`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-08-invalid-credentials-file-f24-status-inva.json | +| F24.status-offline | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Status with an expired login offline (`F24.status-offline`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-18-status-with-an-expired-login-offline-f24.json | +| F24.status-refresh-rejected | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Rejected refresh (`F24.status-refresh-rejected`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-07-rejected-refresh-f24-status-refresh-reje.json | +| F24.status-restart | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: Login across a restart (`F24.status-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-05-login-across-a-restart-f24-status-restar.json | +| F24.ts-client | 20261111-084025-c72f-map-f24 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F24-openai-subscription.md: TypeScript client (`F24.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084025-c72f-map-f24/evidence/map-run/F24-openai-subscription-15-typescript-client-f24-ts-client.json | +| F25.auth | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Session key required (`F25.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-01-session-key-required-f25-auth.json | +| F25.catalog-project | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Project skills (`F25.catalog-project`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-04-project-skills-f25-catalog-project.json | +| F25.catalog-registered-marketplace | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Registered marketplace skills (`F25.catalog-registered-marketplace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-26-registered-marketplace-skills-f25-catalo.json | +| F25.catalog-server-marketplaces | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Server-default marketplaces (`F25.catalog-server-marketplaces`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-30-server-default-marketplaces-f25-catalog-.json | +| F25.catalog-server-sources | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Sandbox and organization sources (`F25.catalog-server-sources`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-06-sandbox-and-organization-sources-f25-cat.json | +| F25.catalog-user-precedence | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: User skills and precedence (`F25.catalog-user-precedence`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-05-user-skills-and-precedence-f25-catalog-u.json | +| F25.catalog-validation | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Request validation (`F25.catalog-validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-07-request-validation-f25-catalog-validatio.json | +| F25.conversation-activation | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Installed skill in a conversation (`F25.conversation-activation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-27-installed-skill-in-a-conversation-f25-co.json | +| F25.enable-disable | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Disable and enable (`F25.enable-disable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-09-disable-and-enable-f25-enable-disable.json | +| F25.install-bad-frontmatter | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Malformed frontmatter (`F25.install-bad-frontmatter`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-12-malformed-frontmatter-f25-install-bad-fr.json | +| F25.install-conflict | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Already installed (`F25.install-conflict`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-10-already-installed-f25-install-conflict.json | +| F25.install-documented-sources | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Documented install sources (`F25.install-documented-sources`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-16-documented-install-sources-f25-install-d.json | +| F25.install-errors | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Install errors (`F25.install-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-11-install-errors-f25-install-errors.json | +| F25.install-git | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Install from a git source (`F25.install-git`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-13-install-from-a-git-source-f25-install-gi.json | +| F25.install-github | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Install from GitHub (`F25.install-github`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-15-install-from-github-f25-install-github.json | +| F25.install-local | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Install from a local directory (`F25.install-local`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-08-install-from-a-local-directory-f25-insta.json | +| F25.install-ref | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Install at a ref (`F25.install-ref`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-18-install-at-a-ref-f25-install-ref.json | +| F25.install-unknown-ref | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Unknown ref (`F25.install-unknown-ref`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-19-unknown-ref-f25-install-unknown-ref.json | +| F25.invoke-skill | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Skill invoked through `invoke_skill` (`F25.invoke-skill`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-28-skill-invoked-through-invoke-skill-f25-i.json | +| F25.marketplace-catalog | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Marketplace catalog (`F25.marketplace-catalog`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-31-marketplace-catalog-f25-marketplace-cata.json | +| F25.marketplace-public | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Public marketplace catalog (`F25.marketplace-public`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-03-public-marketplace-catalog-f25-marketpla.json | +| F25.marketplace-stale-cache | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Catalog after a sync (`F25.marketplace-stale-cache`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-33-catalog-after-a-sync-f25-marketplace-sta.json | +| F25.name-errors | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Unknown and invalid names (`F25.name-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-21-unknown-and-invalid-names-f25-name-error.json | +| F25.profile-catalog | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Profile launch catalog (`F25.profile-catalog`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-24-profile-launch-catalog-f25-profile-catal.json | +| F25.public-repo | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: The real public repository (`F25.public-repo`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-02-the-real-public-repository-f25-public-re.json | +| F25.refresh | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Refresh from the source (`F25.refresh`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-17-refresh-from-the-source-f25-refresh.json | +| F25.refresh-unfetchable | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Refresh of a vanished source (`F25.refresh-unfetchable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-20-refresh-of-a-vanished-source-f25-refresh.json | +| F25.restart-persist | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Restart (`F25.restart-persist`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-29-restart-f25-restart-persist.json | +| F25.sdk-load-skills | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: SDK consumer (`F25.sdk-load-skills`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-14-sdk-consumer-f25-sdk-load-skills.json | +| F25.self-heal | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Hand-copied and hand-deleted directories (`F25.self-heal`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-22-hand-copied-and-hand-deleted-directories.json | +| F25.sync | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Sync the public repository (`F25.sync`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-23-sync-the-public-repository-f25-sync.json | +| F25.sync-offline | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Sync without a reachable origin (`F25.sync-offline`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-25-sync-without-a-reachable-origin-f25-sync.json | +| F25.ts-client | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: TypeScript client (`F25.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-32-typescript-client-f25-ts-client.json | +| F25.uninstall | 20261111-090123-11a5-map-f25 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F25-skills.md: Uninstall (`F25.uninstall`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090123-11a5-map-f25/evidence/map-run/F25-skills-34-uninstall-f25-uninstall.json | +| F26.auth | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Session key required (`F26.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-01-session-key-required-f26-auth.json | +| F26.catalog-project | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Project plugins (`F26.catalog-project`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-02-project-plugins-f26-catalog-project.json | +| F26.catalog-user-precedence | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: User plugins and precedence (`F26.catalog-user-precedence`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-03-user-plugins-and-precedence-f26-catalog-.json | +| F26.conversation-ambient | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Ambient load into a conversation (`F26.conversation-ambient`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-19-ambient-load-into-a-conversation-f26-con.json | +| F26.conversation-disabled | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Disabled plugin in a new conversation (`F26.conversation-disabled`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-20-disabled-plugin-in-a-new-conversation-f2.json | +| F26.enable-disable | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Enable and disable (`F26.enable-disable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-13-enable-and-disable-f26-enable-disable.json | +| F26.fetch-source | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Fetch without installing (`F26.fetch-source`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-05-fetch-without-installing-f26-fetch-sourc.json | +| F26.install-agent-plugins | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Agent Plugins layout (`F26.install-agent-plugins`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-06-agent-plugins-layout-f26-install-agent-p.json | +| F26.install-conflict | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Duplicate install (`F26.install-conflict`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-09-duplicate-install-f26-install-conflict.json | +| F26.install-documented-source | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Documented install source (`F26.install-documented-source`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-12-documented-install-source-f26-install-do.json | +| F26.install-errors | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Install errors (`F26.install-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-10-install-errors-f26-install-errors.json | +| F26.install-git | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Install from a git source (`F26.install-git`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-07-install-from-a-git-source-f26-install-gi.json | +| F26.install-github | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Install from GitHub (`F26.install-github`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-11-install-from-github-f26-install-github.json | +| F26.install-local | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Install from a local path (`F26.install-local`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-04-install-from-a-local-path-f26-install-lo.json | +| F26.install-repo-root-name | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Repository root name (`F26.install-repo-root-name`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-08-repository-root-name-f26-install-repo-ro.json | +| F26.marketplace-cache | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Catalog cache (`F26.marketplace-cache`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-24-catalog-cache-f26-marketplace-cache.json | +| F26.marketplace-local | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Marketplace against a local stand-in (`F26.marketplace-local`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-23-marketplace-against-a-local-stand-in-f26.json | +| F26.marketplace-public | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Public marketplace (`F26.marketplace-public`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-26-public-marketplace-f26-marketplace-publi.json | +| F26.name-errors | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Unknown and invalid names (`F26.name-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-14-unknown-and-invalid-names-f26-name-error.json | +| F26.refresh | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Refresh (`F26.refresh`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-15-refresh-f26-refresh.json | +| F26.refresh-unfetchable | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Unfetchable refresh (`F26.refresh-unfetchable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-17-unfetchable-refresh-f26-refresh-unfetcha.json | +| F26.restart-persist | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Persistence across a restart (`F26.restart-persist`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-22-persistence-across-a-restart-f26-restart.json | +| F26.self-heal | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Hand-edited store (`F26.self-heal`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-16-hand-edited-store-f26-self-heal.json | +| F26.ts-client | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: TypeScript client (`F26.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-25-typescript-client-f26-ts-client.json | +| F26.uninstall | 20261111-090450-9d70-map-f26 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F26-plugins.md: Uninstall (`F26.uninstall`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090450-9d70-map-f26/evidence/map-run/F26-plugins-18-uninstall-f26-uninstall.json | +| F27.agent-definitions | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Forwarded sub-agent definitions (`F27.agent-definitions`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-22-forwarded-sub-agent-definitions-f27-agen.json | +| F27.auth | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Session key required (`F27.auth`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-01-session-key-required-f27-auth.json | +| F27.browser-disabled | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Browser disabled by config (`F27.browser-disabled`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-29-browser-disabled-by-config-f27-browser-d.json | +| F27.browser-disabled-launch | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Default tools follow the browser setting (`F27.browser-disabled-launch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-28-default-tools-follow-the-browser-setting.json | +| F27.catalog | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Tool catalog (`F27.catalog`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-05-tool-catalog-f27-catalog.json | +| F27.catalog-docker-runtime | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Docker conversation runtime (`F27.catalog-docker-runtime`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-32-docker-conversation-runtime-f27-catalog-.json | +| F27.catalog-sealed | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Client tools stay out of the catalog (`F27.catalog-sealed`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-20-client-tools-stay-out-of-the-catalog-f27.json | +| F27.child-conversation-tool | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Child conversation tool (`F27.child-conversation-tool`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-06-child-conversation-tool-f27-child-conver.json | +| F27.client-tool-call | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Client tool call (`F27.client-tool-call`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-21-client-tool-call-f27-client-tool-call.json | +| F27.custom-tool-modules | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Custom tool module (`F27.custom-tool-modules`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-23-custom-tool-module-f27-custom-tool-modul.json | +| F27.custom-tool-modules | 20261111-084831-0bb5-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Custom tool module (`F27.custom-tool-modules`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084831-0bb5-map-f27/evidence/map-run/F27-hooks-subagents-tools-01-custom-tool-module-f27-custom-tool-modul.json | +| F27.custom-tool-modules | 20261111-090008-47b9-map-f27 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F27-hooks-subagents-tools.md: Custom tool module (`F27.custom-tool-modules`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090008-47b9-map-f27/evidence/map-run/F27-hooks-subagents-tools-01-custom-tool-module-f27-custom-tool-modul.json | +| F27.custom-tool-modules | 20261111-090141-d9c4-map-f27 | 391b5546481d511261a4850e458b487478c663f0 | pass | map run F27-hooks-subagents-tools.md: Custom tool module (`F27.custom-tool-modules`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090141-d9c4-map-f27/evidence/map-run/F27-hooks-subagents-tools-01-custom-tool-module-f27-custom-tool-modul.json | +| F27.hooks-absent | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: No hooks (`F27.hooks-absent`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-09-no-hooks-f27-hooks-absent.json | +| F27.hooks-block | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Blocking hooks (`F27.hooks-block`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-19-blocking-hooks-f27-hooks-block.json | +| F27.hooks-conversation | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Hooks in a conversation (`F27.hooks-conversation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-17-hooks-in-a-conversation-f27-hooks-conver.json | +| F27.hooks-formats | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Formats (`F27.hooks-formats`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-08-formats-f27-hooks-formats.json | +| F27.hooks-invalid | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Broken hooks.json (`F27.hooks-invalid`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-10-broken-hooks-json-f27-hooks-invalid.json | +| F27.hooks-not-automatic | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Hooks are not loaded implicitly (`F27.hooks-not-automatic`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-18-hooks-are-not-loaded-implicitly-f27-hook.json | +| F27.hooks-project | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Project hooks (`F27.hooks-project`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-07-project-hooks-f27-hooks-project.json | +| F27.server-info-browser-disabled | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Usable tools with the browser disabled (`F27.server-info-browser-disabled`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-30-usable-tools-with-the-browser-disabled-f.json | +| F27.subagents-bad-skill | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Agent with a missing skill (`F27.subagents-bad-skill`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-16-agent-with-a-missing-skill-f27-subagents.json | +| F27.subagents-browser-disabled | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Browser-only sub-agent with the browser disabled (`F27.subagents-browser-disabled`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-31-browser-only-sub-agent-with-the-browser-.json | +| F27.subagents-builtin | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Sub-agent built-ins (`F27.subagents-builtin`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-11-sub-agent-built-ins-f27-subagents-builti.json | +| F27.subagents-invalid | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Invalid definitions (`F27.subagents-invalid`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-15-invalid-definitions-f27-subagents-invali.json | +| F27.subagents-precedence | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Precedence (`F27.subagents-precedence`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-14-precedence-f27-subagents-precedence.json | +| F27.subagents-project | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Project sub-agents (`F27.subagents-project`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-12-project-sub-agents-f27-subagents-project.json | +| F27.subagents-user | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: User sub-agents (`F27.subagents-user`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-13-user-sub-agents-f27-subagents-user.json | +| F27.tools-list | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Registered tools (`F27.tools-list`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-03-registered-tools-f27-tools-list.json | +| F27.tools-redirect | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Slash redirects (`F27.tools-redirect`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-04-slash-redirects-f27-tools-redirect.json | +| F27.validation | 20261111-084114-3c93-map-f27 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F27-hooks-subagents-tools.md: Bad bodies (`F27.validation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084114-3c93-map-f27/evidence/map-run/F27-hooks-subagents-tools-02-bad-bodies-f27-validation.json | +| F28.backend-approval-gate | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Approval gate (`F28.backend-approval-gate`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-13-approval-gate-f28-backend-approval-gate.json | +| F28.backend-inherit-environment | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Inherited environment (`F28.backend-inherit-environment`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-21-inherited-environment-f28-backend-inheri.json | +| F28.backend-logs | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Logs while starting (`F28.backend-logs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-18-logs-while-starting-f28-backend-logs.json | +| F28.backend-prepare | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Prepare (`F28.backend-prepare`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-14-prepare-f28-backend-prepare.json | +| F28.backend-secret-grant | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Secret grant (`F28.backend-secret-grant`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-30-secret-grant-f28-backend-secret-grant.json | +| F28.backend-start | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Start (`F28.backend-start`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-15-start-f28-backend-start.json | +| F28.backend-start-fails | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Failed starts (`F28.backend-start-fails`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-20-failed-starts-f28-backend-start-fails.json | +| F28.backend-status | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Backend status (`F28.backend-status`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-12-backend-status-f28-backend-status.json | +| F28.backend-stop | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Stop (`F28.backend-stop`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-16-stop-f28-backend-stop.json | +| F28.broken-install | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Broken installs (`F28.broken-install`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-11-broken-installs-f28-broken-install.json | +| F28.bundle | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Bundle (`F28.bundle`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-06-bundle-f28-bundle.json | +| F28.disable-stops-backend | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Disable stops the backend (`F28.disable-stops-backend`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-22-disable-stops-the-backend-f28-disable-st.json | +| F28.enable-disable | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Enable and disable (`F28.enable-disable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-08-enable-and-disable-f28-enable-disable.json | +| F28.force-reinstall | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Reinstall (`F28.force-reinstall`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-17-reinstall-f28-force-reinstall.json | +| F28.hard-restart-orphan | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Backend outlives a crash (`F28.hard-restart-orphan`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-29-backend-outlives-a-crash-f28-hard-restar.json | +| F28.icon | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Icon (`F28.icon`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-07-icon-f28-icon.json | +| F28.install-errors | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Install errors (`F28.install-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-05-install-errors-f28-install-errors.json | +| F28.install-git | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Install from a git URL (`F28.install-git`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-02-install-from-a-git-url-f28-install-git.json | +| F28.install-local | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Install from a local directory (`F28.install-local`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-01-install-from-a-local-directory-f28-insta.json | +| F28.install-ref | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Pin a git ref (`F28.install-ref`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-03-pin-a-git-ref-f28-install-ref.json | +| F28.install-ref-cached | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Unknown ref on a cached source (`F28.install-ref-cached`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-26-unknown-ref-on-a-cached-source-f28-insta.json | +| F28.install-repo-path | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Install a subdirectory (`F28.install-repo-path`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-04-install-a-subdirectory-f28-install-repo-.json | +| F28.list-discovery | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Discovery by listing (`F28.list-discovery`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-10-discovery-by-listing-f28-list-discovery.json | +| F28.prepare-corrupt-archive | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Non-gzip artifact (`F28.prepare-corrupt-archive`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-25-non-gzip-artifact-f28-prepare-corrupt-ar.json | +| F28.restart-persistence | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Persistence across a restart (`F28.restart-persistence`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-23-persistence-across-a-restart-f28-restart.json | +| F28.status-exit-hang | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Status after the leader exits (`F28.status-exit-hang`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-28-status-after-the-leader-exits-f28-status.json | +| F28.status-probe-latch | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Health probe latch (`F28.status-probe-latch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-27-health-probe-latch-f28-status-probe-latc.json | +| F28.ts-client | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: TypeScript client (`F28.ts-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-19-typescript-client-f28-ts-client.json | +| F28.uninstall | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Uninstall (`F28.uninstall`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-24-uninstall-f28-uninstall.json | +| F28.unknown-name | 20261111-090539-d42d-map-f28 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F28-canvas-extensions.md: Unknown and invalid names (`F28.unknown-name`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090539-d42d-map-f28/evidence/map-run/F28-canvas-extensions-09-unknown-and-invalid-names-f28-unknown-na.json | +| F29.backend-crash-detected | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Crash seen by the bridge (`F29.backend-crash-detected`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-24-crash-seen-by-the-bridge-f29-backend-cra.json | +| F29.backend-lifecycle-revokes | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Backend stop and disable revoke (`F29.backend-lifecycle-revokes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-23-backend-stop-and-disable-revoke-f29-back.json | +| F29.ingress-advertised | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Ingress advertised (`F29.ingress-advertised`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-01-ingress-advertised-f29-ingress-advertise.json | +| F29.ingress-config-errors | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Ingress not configured or unusable (`F29.ingress-config-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-02-ingress-not-configured-or-unusable-f29-i.json | +| F29.proxy-cookie-required | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Cookie required (`F29.proxy-cookie-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-10-cookie-required-f29-proxy-cookie-require.json | +| F29.proxy-credentials-stripped | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Credentials stripped (`F29.proxy-credentials-stripped`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-14-credentials-stripped-f29-proxy-credentia.json | +| F29.proxy-get | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Proxied safe methods (`F29.proxy-get`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-09-proxied-safe-methods-f29-proxy-get.json | +| F29.proxy-path-traversal | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Path traversal (`F29.proxy-path-traversal`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-18-path-traversal-f29-proxy-path-traversal.json | +| F29.proxy-request-framing | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Request framing (`F29.proxy-request-framing`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-15-request-framing-f29-proxy-request-framin.json | +| F29.proxy-response-filter | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Response filter (`F29.proxy-response-filter`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-16-response-filter-f29-proxy-response-filte.json | +| F29.proxy-response-headers | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: One Date and Server header (`F29.proxy-response-headers`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-17-one-date-and-server-header-f29-proxy-res.json | +| F29.proxy-root | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: App root (`F29.proxy-root`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-12-app-root-f29-proxy-root.json | +| F29.proxy-root-path-query | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Root `path` parameter (`F29.proxy-root-path-query`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-13-root-path-parameter-f29-proxy-root-path-.json | +| F29.proxy-unsafe-methods | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Unsafe methods (`F29.proxy-unsafe-methods`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-11-unsafe-methods-f29-proxy-unsafe-methods.json | +| F29.session-backend-not-ready | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Backend not ready (`F29.session-backend-not-ready`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-08-backend-not-ready-f29-session-backend-no.json | +| F29.session-bootstrap | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Session bootstrap (`F29.session-bootstrap`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-03-session-bootstrap-f29-session-bootstrap.json | +| F29.session-configured-control-origin | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Configured control origins (`F29.session-configured-control-origin`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-25-configured-control-origins-f29-session-c.json | +| F29.session-forwarded-proto | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Forwarded proto (`F29.session-forwarded-proto`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-06-forwarded-proto-f29-session-forwarded-pr.json | +| F29.session-ingress-host | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Ingress host (`F29.session-ingress-host`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-05-ingress-host-f29-session-ingress-host.json | +| F29.session-key-required | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Session key required (`F29.session-key-required`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-04-session-key-required-f29-session-key-req.json | +| F29.session-origin-rules | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Origin rules (`F29.session-origin-rules`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-07-origin-rules-f29-session-origin-rules.json | +| F29.session-revoke | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Revoke a session (`F29.session-revoke`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-22-revoke-a-session-f29-session-revoke.json | +| F29.session-ttl | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Session expiry (`F29.session-ttl`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-28-session-expiry-f29-session-ttl.json | +| F29.sessions-not-persisted | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Server restart (`F29.sessions-not-persisted`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-27-server-restart-f29-sessions-not-persiste.json | +| F29.trusted-forwarded-headers | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Trusted forwarded headers (`F29.trusted-forwarded-headers`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-26-trusted-forwarded-headers-f29-trusted-fo.json | +| F29.uninstall-revokes | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Uninstall (`F29.uninstall-revokes`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-29-uninstall-f29-uninstall-revokes.json | +| F29.ws-bridge | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: WebSocket bridge (`F29.ws-bridge`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-19-websocket-bridge-f29-ws-bridge.json | +| F29.ws-handshake-rejections | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Handshake rejections (`F29.ws-handshake-rejections`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-20-handshake-rejections-f29-ws-handshake-re.json | +| F29.ws-subprotocol | 20261111-084215-ec44-map-f29 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F29-app-backend-bridge.md: Subprotocol (`F29.ws-subprotocol`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084215-ec44-map-f29/evidence/map-run/F29-app-backend-bridge-21-subprotocol-f29-ws-subprotocol.json | +| F30.active-agent-profile | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Active Agent Profile (`F30.active-agent-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-06-active-agent-profile-f30-active-agent-pr.json | +| F30.auth-bearer-or-header | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Bearer or session header (`F30.auth-bearer-or-header`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-02-bearer-or-session-header-f30-auth-bearer.json | +| F30.auth-open-without-keys | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: No session keys (`F30.auth-open-without-keys`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-03-no-session-keys-f30-auth-open-without-ke.json | +| F30.chat-agent-tools | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: The agent runs its own tools (`F30.chat-agent-tools`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-16-the-agent-runs-its-own-tools-f30-chat-ag.json | +| F30.chat-basic | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Chat completion (`F30.chat-basic`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-04-chat-completion-f30-chat-basic.json | +| F30.chat-confirmation-409 | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Confirmation stops a new conversation (`F30.chat-confirmation-409`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-20-confirmation-stops-a-new-conversation-f3.json | +| F30.chat-header-new-id | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Caller-chosen id (`F30.chat-header-new-id`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-12-caller-chosen-id-f30-chat-header-new-id.json | +| F30.chat-input-errors | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Chat input errors (`F30.chat-input-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-17-chat-input-errors-f30-chat-input-errors.json | +| F30.chat-latest-user-only | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Only the latest user turn (`F30.chat-latest-user-only`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-08-only-the-latest-user-turn-f30-chat-lates.json | +| F30.chat-reuse-after-restart | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Continue after a restart (`F30.chat-reuse-after-restart`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-11-continue-after-a-restart-f30-chat-reuse-.json | +| F30.chat-reuse-confirmation | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Confirmation on a continued conversation (`F30.chat-reuse-confirmation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-21-confirmation-on-a-continued-conversation.json | +| F30.chat-reuse-header | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Continue with the header (`F30.chat-reuse-header`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-10-continue-with-the-header-f30-chat-reuse-.json | +| F30.chat-reuse-keeps-llm | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Continuing keeps the conversation's LLM (`F30.chat-reuse-keeps-llm`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-13-continuing-keeps-the-conversation-s-llm-.json | +| F30.chat-reuse-run-limit-429 | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Run limit on a continued conversation (`F30.chat-reuse-run-limit-429`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-23-run-limit-on-a-continued-conversation-f3.json | +| F30.chat-run-error-500 | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Run ends in error (`F30.chat-run-error-500`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-19-run-ends-in-error-f30-chat-run-error-500.json | +| F30.chat-run-limit-429 | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Run limit (`F30.chat-run-limit-429`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-22-run-limit-f30-chat-run-limit-429.json | +| F30.chat-stream | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Streaming (`F30.chat-stream`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-09-streaming-f30-chat-stream.json | +| F30.chat-system-suffix | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: System text goes to the system prompt (`F30.chat-system-suffix`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-07-system-text-goes-to-the-system-prompt-f3.json | +| F30.chat-timeout-504 | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Gateway timeout (`F30.chat-timeout-504`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-24-gateway-timeout-f30-chat-timeout-504.json | +| F30.chat-usage-per-request | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Usage per request (`F30.chat-usage-per-request`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-14-usage-per-request-f30-chat-usage-per-req.json | +| F30.inner-gateway-run | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Inner gateway run (`F30.inner-gateway-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-05-inner-gateway-run-f30-inner-gateway-run.json | +| F30.models-list | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Model list (`F30.models-list`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-01-model-list-f30-models-list.json | +| F30.observability-headers | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Observability headers (`F30.observability-headers`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-18-observability-headers-f30-observability-.json | +| F30.openai-sdk | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Official OpenAI SDK (`F30.openai-sdk`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-28-official-openai-sdk-f30-openai-sdk.json | +| F30.responses-basic | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Responses (`F30.responses-basic`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-25-responses-f30-responses-basic.json | +| F30.responses-errors | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Responses errors (`F30.responses-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-27-responses-errors-f30-responses-errors.json | +| F30.responses-stateless-replay | 20261111-084225-4d3d-map-f30 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F30-openai-gateway.md: Stateless replay (`F30.responses-stateless-replay`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084225-4d3d-map-f30/evidence/map-run/F30-openai-gateway-26-stateless-replay-f30-responses-stateless.json | +| F31.config-invalid-fails-fast | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Invalid limits stop startup (`F31.config-invalid-fails-fast`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-20-invalid-limits-stop-startup-f31-config-i.json | +| F31.config-null-override | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Null from the environment (`F31.config-null-override`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-19-null-from-the-environment-f31-config-nul.json | +| F31.config-precedence | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Config file and environment (`F31.config-precedence`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-18-config-file-and-environment-f31-config-p.json | +| F31.config-silent-env | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Unrecognised booleans and choices start anyway (`F31.config-silent-env`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-21-unrecognised-booleans-and-choices-start-.json | +| F31.crash-tool-call | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Crash during a tool call (`F31.crash-tool-call`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-05-crash-during-a-tool-call-f31-crash-tool-.json | +| F31.create-capacity | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Create at capacity (`F31.create-capacity`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-17-create-at-capacity-f31-create-capacity.json | +| F31.eviction-skips-busy | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Busy conversations are kept (`F31.eviction-skips-busy`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-16-busy-conversations-are-kept-f31-eviction.json | +| F31.fenced-write-no-trace | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Refused write leaves no trace (`F31.fenced-write-no-trace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-13-refused-write-leaves-no-trace-f31-fenced.json | +| F31.idle-eviction | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Idle eviction (`F31.idle-eviction`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-15-idle-eviction-f31-idle-eviction.json | +| F31.lease-claimed | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Lease claimed on load (`F31.lease-claimed`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-01-lease-claimed-on-load-f31-lease-claimed.json | +| F31.lease-dead-owner-takeover | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Dead owner takeover (`F31.lease-dead-owner-takeover`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-04-dead-owner-takeover-f31-lease-dead-owner.json | +| F31.lease-disabled | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Leasing disabled (`F31.lease-disabled`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-14-leasing-disabled-f31-lease-disabled.json | +| F31.lease-expiry-fencing | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Lease expiry fences a frozen owner (`F31.lease-expiry-fencing`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-12-lease-expiry-fences-a-frozen-owner-f31-l.json | +| F31.lease-released-graceful | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Graceful release (`F31.lease-released-graceful`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-03-graceful-release-f31-lease-released-grac.json | +| F31.lease-renewed | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Lease renewed (`F31.lease-renewed`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-02-lease-renewed-f31-lease-renewed.json | +| F31.no-plaintext-at-rest | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Nothing secret in plaintext (`F31.no-plaintext-at-rest`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-07-nothing-secret-in-plaintext-f31-no-plain.json | +| F31.second-instance-catalog | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Second server reads the catalog (`F31.second-instance-catalog`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-08-second-server-reads-the-catalog-f31-seco.json | +| F31.second-instance-lease-held | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Second server refused while the owner lives (`F31.second-instance-lease-held`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-09-second-server-refused-while-the-owner-li.json | +| F31.second-instance-takeover | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Takeover after the owner dies (`F31.second-instance-takeover`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-10-takeover-after-the-owner-dies-f31-second.json | +| F31.state-survives-restarts | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: State survives restarts (`F31.state-survives-restarts`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-06-state-survives-restarts-f31-state-surviv.json | +| F31.tmux-dir-cleanup | 20261111-084630-31c6-map-f31 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F31-persistence-and-limits.md: Per-process tmux directory removed on stop (`F31.tmux-dir-cleanup`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084630-31c6-map-f31/evidence/map-run/F31-persistence-and-limits-22-per-process-tmux-directory-removed-on-st.json | +| F32.backpressure | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Batch and queue limits (`F32.backpressure`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-12-batch-and-queue-limits-f32-backpressure.json | +| F32.buffer-size | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Custom headers, path prefix and buffer size (`F32.custom-headers`, `F32.buffer-size`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-07-custom-headers-path-prefix-and-buffer-si.json | +| F32.buffer-stranded | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Events published during an in-flight batch (`F32.buffer-stranded`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-08-events-published-during-an-in-flight-bat.json | +| F32.conversation-created | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Conversation created (`F32.conversation-created`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-03-conversation-created-f32-conversation-cr.json | +| F32.conversation-lifecycle | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Conversation lifecycle posts (`F32.conversation-lifecycle`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-05-conversation-lifecycle-posts-f32-convers.json | +| F32.custom-headers | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Custom headers, path prefix and buffer size (`F32.custom-headers`, `F32.buffer-size`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-07-custom-headers-path-prefix-and-buffer-si.json | +| F32.deferred-init | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Deferred pod gets telemetry and webhooks from `/api/init` (`F32.deferred-init`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-28-deferred-pod-gets-telemetry-and-webhooks.json | +| F32.env-config | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Webhooks from the environment (`F32.env-config`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-16-webhooks-from-the-environment-f32-env-co.json | +| F32.events-full-state | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Full-state snapshot on every attach (`F32.events-full-state`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-04-full-state-snapshot-on-every-attach-f32-.json | +| F32.events-no-deltas | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: No streaming deltas on webhooks (`F32.events-no-deltas`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-06-no-streaming-deltas-on-webhooks-f32-even.json | +| F32.events-post | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: A run's events reach the webhook (`F32.events-post`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-01-a-run-s-events-reach-the-webhook-f32-eve.json | +| F32.events-session-key | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Session key and content type on every request (`F32.events-session-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-02-session-key-and-content-type-on-every-re.json | +| F32.flush-delay | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Flush timer from the first event (`F32.flush-delay`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-10-flush-timer-from-the-first-event-f32-flu.json | +| F32.flush-on-close | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Flush on delete and on shutdown (`F32.flush-on-close`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-09-flush-on-delete-and-on-shutdown-f32-flus.json | +| F32.requeue | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Failed batches are requeued (`F32.requeue`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-14-failed-batches-are-requeued-f32-requeue.json | +| F32.retry | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Retries against a failing receiver (`F32.retry`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-13-retries-against-a-failing-receiver-f32-r.json | +| F32.telemetry-consent-settings | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Consent from settings (`F32.telemetry-consent-settings`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-23-consent-from-settings-f32-telemetry-cons.json | +| F32.telemetry-conversation | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Conversation telemetry is sanitized (`F32.telemetry-conversation`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-19-conversation-telemetry-is-sanitized-f32-.json | +| F32.telemetry-conversation-failed | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Failed runs (`F32.telemetry-conversation-failed`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-22-failed-runs-f32-telemetry-conversation-f.json | +| F32.telemetry-env-override | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Environment override beats settings (`F32.telemetry-env-override`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-24-environment-override-beats-settings-f32-.json | +| F32.telemetry-kill-switch | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Kill switch (`F32.telemetry-kill-switch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-25-kill-switch-f32-telemetry-kill-switch.json | +| F32.telemetry-lifecycle | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Server lifecycle events (`F32.telemetry-lifecycle`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-18-server-lifecycle-events-f32-telemetry-li.json | +| F32.telemetry-off-by-default | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Telemetry is off by default (`F32.telemetry-off-by-default`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-17-telemetry-is-off-by-default-f32-telemetr.json | +| F32.telemetry-request-failed | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Unhandled errors (`F32.telemetry-request-failed`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-20-unhandled-errors-f32-telemetry-request-f.json | +| F32.telemetry-salt-and-kind | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Pseudonym key and deployment kind (`F32.telemetry-salt-and-kind`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-26-pseudonym-key-and-deployment-kind-f32-te.json | +| F32.telemetry-salt-docs | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Salt description (`F32.telemetry-salt-docs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-27-salt-description-f32-telemetry-salt-docs.json | +| F32.telemetry-usage-buckets | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Token and cost buckets from real usage (`F32.telemetry-usage-buckets`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-21-token-and-cost-buckets-from-real-usage-f.json | +| F32.webhook-spec-docs | 20261111-084841-fa4c-map-f32 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F32-webhooks-and-telemetry.md: Webhook field descriptions (`F32.webhook-spec-docs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-084841-fa4c-map-f32/evidence/map-run/F32-webhooks-and-telemetry-11-webhook-field-descriptions-f32-webhook-s.json | +| F33.async-workspace | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Async workspace (`F33.async-workspace`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-07-async-workspace-f33-async-workspace.json | +| F33.client-tools | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Client-defined tool (`F33.client-tools`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-15-client-defined-tool-f33-client-tools.json | +| F33.completion-logs | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Completion logs on the client (`F33.completion-logs`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-19-completion-logs-on-the-client-f33-comple.json | +| F33.conversation-ask-title | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Ask, generate and set a title (`F33.conversation-ask-title`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-23-ask-generate-and-set-a-title-f33-convers.json | +| F33.conversation-attach | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Reattach from a second program (`F33.conversation-attach`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-20-reattach-from-a-second-program-f33-conve.json | +| F33.conversation-attach-large | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Attach to a long history (`F33.conversation-attach-large`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-21-attach-to-a-long-history-f33-conversatio.json | +| F33.conversation-callbacks | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Send and run to completion (`F33.conversation-run`, `F33.conversation-callbacks`, `F33.conversation-stats`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-13-send-and-run-to-completion-f33-conversat.json | +| F33.conversation-close | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Close keeps or deletes (`F33.conversation-close`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-41-close-keeps-or-deletes-f33-conversation-.json | +| F33.conversation-create | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Create a conversation without running it (`F33.conversation-create`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-08-create-a-conversation-without-running-it.json | +| F33.conversation-create-profile | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Create from a saved agent profile (`F33.conversation-create-profile`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-10-create-from-a-saved-agent-profile-f33-co.json | +| F33.conversation-create-request | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Create from a request, twice (`F33.conversation-create-request`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-09-create-from-a-request-twice-f33-conversa.json | +| F33.conversation-fork | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Fork (`F33.conversation-fork`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-24-fork-f33-conversation-fork.json | +| F33.conversation-fork-branch | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Fork from an event, then move HEAD (`F33.conversation-fork-branch`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-25-fork-from-an-event-then-move-head-f33-co.json | +| F33.conversation-interrupt | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Interrupt a blocking run (`F33.conversation-interrupt`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-29-interrupt-a-blocking-run-f33-conversatio.json | +| F33.conversation-plugins-tag | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Plugin source with inline credentials (`F33.conversation-plugins-tag`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-26-plugin-source-with-inline-credentials-f3.json | +| F33.conversation-reconnect-callbacks | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Callbacks after a reconnect (`F33.conversation-reconnect-callbacks`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-37-callbacks-after-a-reconnect-f33-conversa.json | +| F33.conversation-run | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Send and run to completion (`F33.conversation-run`, `F33.conversation-callbacks`, `F33.conversation-stats`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-13-send-and-run-to-completion-f33-conversat.json | +| F33.conversation-run-error | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Provider error surfaces (`F33.conversation-run-error`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-34-provider-error-surfaces-f33-conversation.json | +| F33.conversation-run-join-long-tool | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Join a run during a long tool call (`F33.conversation-run-join-long-tool`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-31-join-a-run-during-a-long-tool-call-f33-c.json | +| F33.conversation-state-reread | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Read the state twice (`F33.conversation-state-reread`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-22-read-the-state-twice-f33-conversation-st.json | +| F33.conversation-stats | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Send and run to completion (`F33.conversation-run`, `F33.conversation-callbacks`, `F33.conversation-stats`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-13-send-and-run-to-completion-f33-conversat.json | +| F33.conversation-stop-hook | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: A Stop hook that refuses the first stop (`F33.conversation-stop-hook`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-14-a-stop-hook-that-refuses-the-first-stop-.json | +| F33.conversation-streaming | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Streamed tokens (`F33.conversation-streaming`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-16-streamed-tokens-f33-conversation-streami.json | +| F33.conversation-streaming-state | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Deltas kept out of `state.events` (`F33.conversation-streaming-state`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-18-deltas-kept-out-of-state-events-f33-conv.json | +| F33.conversation-stuck | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: A stuck agent (`F33.conversation-stuck`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-32-a-stuck-agent-f33-conversation-stuck.json | +| F33.conversation-token-callbacks | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Token callbacks on a remote conversation (`F33.conversation-token-callbacks`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-17-token-callbacks-on-a-remote-conversation.json | +| F33.custom-tool-module-unimportable | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: An agent tool the server cannot import (`F33.custom-tool-module-unimportable`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-40-an-agent-tool-the-server-cannot-import-f.json | +| F33.workspace-background | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Background command (`F33.workspace-background`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-03-background-command-f33-workspace-backgro.json | +| F33.workspace-bad-key | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Wrong session key (`F33.workspace-bad-key`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-02-wrong-session-key-f33-workspace-bad-key.json | +| F33.workspace-exec | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Commands, server info and liveness (`F33.workspace-exec`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-01-commands-server-info-and-liveness-f33-wo.json | +| F33.workspace-files | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: File round trip (`F33.workspace-files`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-04-file-round-trip-f33-workspace-files.json | +| F33.workspace-git | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Git changes and diff (`F33.workspace-git`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-05-git-changes-and-diff-f33-workspace-git.json | +| F33.workspace-runtime-scope | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: Conversation-scoped workspace (`F33.workspace-runtime-scope`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-11-conversation-scoped-workspace-f33-worksp.json | +| F33.workspace-settings | 20261111-090619-070e-map-f33 | e776f878876e902a96f4d81bdf20e5a9e7af52bd | pass | map run F33-sdk-remote-client.md: LLM profiles, secrets and MCP from the server (`F33.workspace-settings`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-090619-070e-map-f33/evidence/map-run/F33-sdk-remote-client-06-llm-profiles-secrets-and-mcp-from-the-se.json | +| F34.bash-client | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Bash client (`F34.bash-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-09-bash-client-f34-bash-client.json | +| F34.bash-websocket | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Bash events socket from Node (`F34.bash-websocket`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-10-bash-events-socket-from-node-f34-bash-we.json | +| F34.conversation-branching | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Fork, navigate and ask (`F34.conversation-branching`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-24-fork-navigate-and-ask-f34-conversation-b.json | +| F34.conversation-confirm | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Confirmation and secrets (`F34.conversation-confirm`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-25-confirmation-and-secrets-f34-conversatio.json | +| F34.conversation-controls | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Title and confirmation policy (`F34.conversation-controls`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-19-title-and-confirmation-policy-f34-conver.json | +| F34.conversation-create | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Create a conversation (`F34.conversation-create`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-12-create-a-conversation-f34-conversation-c.json | +| F34.conversation-run | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Send and run (`F34.conversation-run`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-13-send-and-run-f34-conversation-run.json | +| F34.endpoint-audit | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Endpoint audit against this checkout (`F34.endpoint-audit`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-31-endpoint-audit-against-this-checkout-f34.json | +| F34.event-stream | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Event stream on Node's WebSocket (`F34.event-stream`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-17-event-stream-on-node-s-websocket-f34-eve.json | +| F34.event-stream-reconnect | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Event stream reconnect (`F34.event-stream-reconnect`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-29-event-stream-reconnect-f34-event-stream-.json | +| F34.event-stream-reconnect-native | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Reconnect on Node's own WebSocket (`F34.event-stream-reconnect-native`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-30-reconnect-on-node-s-own-websocket-f34-ev.json | +| F34.events-list | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Events list (`F34.events-list`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-14-events-list-f34-events-list.json | +| F34.events-search-kind | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Short event kinds (`F34.events-search-kind`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-16-short-event-kinds-f34-events-search-kind.json | +| F34.events-unknown-id | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Unknown event id (`F34.events-unknown-id`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-15-unknown-event-id-f34-events-unknown-id.json | +| F34.hooks-client | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Hooks (`F34.hooks-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-26-hooks-f34-hooks-client.json | +| F34.http-errors | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Errors (`F34.http-errors`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-02-errors-f34-http-errors.json | +| F34.manager | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Conversation manager (`F34.manager`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-20-conversation-manager-f34-manager.json | +| F34.manager-acp | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: ACP conversations (`F34.manager-acp`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-23-acp-conversations-f34-manager-acp.json | +| F34.manager-create-options | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Create options (`F34.manager-create-options`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-22-create-options-f34-manager-create-option.json | +| F34.manager-update-result | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Update result (`F34.manager-update-result`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-21-update-result-f34-manager-update-result.json | +| F34.mcp-client | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: MCP test (`F34.mcp-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-27-mcp-test-f34-mcp-client.json | +| F34.restart-reload | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Restart (`F34.restart-reload`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-28-restart-f34-restart-reload.json | +| F34.server-client | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Server status and version gate (`F34.server-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-01-server-status-and-version-gate-f34-serve.json | +| F34.version-gate-old | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Old server version (`F34.version-gate-old`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-32-old-server-version-f34-version-gate-old.json | +| F34.workspace-exec | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Workspace commands (`F34.workspace-exec`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-03-workspace-commands-f34-workspace-exec.json | +| F34.workspace-files | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Workspace files (`F34.workspace-files`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-04-workspace-files-f34-workspace-files.json | +| F34.workspace-git | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Git through RemoteWorkspace (`F34.workspace-git`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-07-git-through-remoteworkspace-f34-workspac.json | +| F34.workspace-git-types | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Git status type (`F34.workspace-git-types`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-08-git-status-type-f34-workspace-git-types.json | +| F34.workspace-scoped | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Conversation-scoped workspace (`F34.workspace-scoped`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-06-conversation-scoped-workspace-f34-worksp.json | +| F34.workspace-session | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Workspace session (`F34.workspace-session`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-11-workspace-session-f34-workspace-session.json | +| F34.ws-callback-client | 20261111-085047-3a83-map-f34 | faeba5ef369c23258b5538059337c5e9119c6c05 | pass | map run F34-typescript-client.md: Conversation callback socket from Node (`F34.ws-callback-client`). | every command in the bullet exits 0 (its --expect/--check assertions hold) | exit 0 | /agent-server/20261111-085047-3a83-map-f34/evidence/map-run/F34-typescript-client-18-conversation-callback-socket-from-node-f.json | diff --git a/.pr/map-maintenance/2026-11-11/evidence/f07-full-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f07-full-ledger.jsonl new file mode 100644 index 00000000000..b7d1d05f830 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f07-full-ledger.jsonl @@ -0,0 +1,24 @@ +{"at": "2026-11-11T08:26:34Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.sync-first", "result": "pass", "entry": "map run F07-session-socket.md: Sync comes first (`F07.sync-first`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-01-sync-comes-first-f07-sync-first.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:26:37Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.snapshot-transient", "result": "pass", "entry": "map run F07-session-socket.md: State snapshot (`F07.snapshot-transient`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-02-state-snapshot-f07-snapshot-transient.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:26:46Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.full-replay", "result": "pass", "entry": "map run F07-session-socket.md: Full replay (`F07.full-replay`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-03-full-replay-f07-full-replay.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:26:47Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.seq-on-disk", "result": "pass", "entry": "map run F07-session-socket.md: Seq is the on-disk index (`F07.seq-on-disk`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-04-seq-is-the-on-disk-index-f07-seq-on-disk.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:26:56Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.resume-cursor", "result": "pass", "entry": "map run F07-session-socket.md: Resume from a cursor (`F07.resume-cursor`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-05-resume-from-a-cursor-f07-resume-cursor.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:04Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.error-frame", "result": "pass", "entry": "map run F07-session-socket.md: Invalid message (`F07.error-frame`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-06-invalid-message-f07-error-frame.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:06Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.error-non-json", "result": "pass", "entry": "map run F07-session-socket.md: Non-JSON and binary frames (`F07.error-non-json`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-07-non-json-and-binary-frames-f07-error-non.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:12Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.reject-non-user", "result": "fail", "entry": "map run F07-session-socket.md: Non-user role (`F07.reject-non-user`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F07-session-socket-08-non-user-role-f07-reject-non-user.json"], "note": "automated map run; prose-only expectations were not re-judged; failed at line 3: control-agent-server state cat \"evidence/F07.reject-non-user/assistant-$CID.json\" --check frames.4.frame.type eq error --check frames.4.frame.detail contains 'Only user messages are allowed'"} +{"at": "2026-11-11T08:27:19Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.auth-frame-ignored", "result": "pass", "entry": "map run F07-session-socket.md: Redundant auth frame (`F07.auth-frame-ignored`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-09-redundant-auth-frame-f07-auth-frame-igno.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:22Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.auth", "result": "pass", "entry": "map run F07-session-socket.md: Authentication (`F07.auth`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-10-authentication-f07-auth.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:23Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.unknown-conversation", "result": "pass", "entry": "map run F07-session-socket.md: Unknown conversation (`F07.unknown-conversation`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-11-unknown-conversation-f07-unknown-convers.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:24Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.bad-handshake", "result": "pass", "entry": "map run F07-session-socket.md: Malformed handshake (`F07.bad-handshake`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-12-malformed-handshake-f07-bad-handshake.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:31Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.live-durable", "result": "pass", "entry": "map run F07-session-socket.md: Live durable frame (`F07.live-durable`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-13-live-durable-frame-f07-live-durable.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:37Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.live-order", "result": "pass", "entry": "map run F07-session-socket.md: Live frames in seq order (`F07.live-order`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-14-live-frames-in-seq-order-f07-live-order.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:42Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.replay-then-live", "result": "pass", "entry": "map run F07-session-socket.md: Replay and live tail, each event once (`F07.replay-then-live`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-15-replay-and-live-tail-each-event-once-f07.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:27:48Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.send-message", "result": "pass", "entry": "map run F07-session-socket.md: Send a message on the socket (`F07.send-message`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-16-send-a-message-on-the-socket-f07-send-me.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:28:03Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.restart-resume", "result": "pass", "entry": "map run F07-session-socket.md: Cursor survives a restart (`F07.restart-resume`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-17-cursor-survives-a-restart-f07-restart-re.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:28:07Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.progress-frames", "result": "pass", "entry": "map run F07-session-socket.md: Streaming progress frames (`F07.progress-frames`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-18-streaming-progress-frames-f07-progress-f.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:28:10Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.progress-not-replayed", "result": "pass", "entry": "map run F07-session-socket.md: Progress is never replayed (`F07.progress-not-replayed`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-19-progress-is-never-replayed-f07-progress-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:28:22Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.item-aborted", "result": "pass", "entry": "map run F07-session-socket.md: Interrupted stream (`F07.item-aborted`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-20-interrupted-stream-f07-item-aborted.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:28:29Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.frame-too-large", "result": "pass", "entry": "map run F07-session-socket.md: Oversized event (`F07.frame-too-large`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-21-oversized-event-f07-frame-too-large.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:28:44Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.slow-consumer", "result": "pass", "entry": "map run F07-session-socket.md: Stalled reader (`F07.slow-consumer`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-22-stalled-reader-f07-slow-consumer.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:28:45Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.subscriber-limit", "result": "pass", "entry": "map run F07-session-socket.md: Subscriber cap (`F07.subscriber-limit`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F07-session-socket-23-subscriber-cap-f07-subscriber-limit.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:28:52Z", "run": "20261111-082602-7f35-map-f07", "checkout_sha": "5ff99f1b6ca28f7d16870b69f57a31351e59bcd0", "feature": "F07.deleted-close", "result": "xfail", "entry": "map run F07-session-socket.md: Deleting closes the socket (`F07.deleted-close`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F07-session-socket-24-deleting-closes-the-socket-f07-deleted-c.json"], "note": "known bug reproduced (expected failure); failed at line 7: control-agent-server assert --check close.code exists"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f07-sync-first-during-run-1-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f07-sync-first-during-run-1-ledger.jsonl new file mode 100644 index 00000000000..40f82f3ae03 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f07-sync-first-during-run-1-ledger.jsonl @@ -0,0 +1 @@ +{"at": "2026-11-11T09:12:18Z", "run": "20261111-091133-869f-map-f07", "checkout_sha": "edff6711505acb0f88d90dbbaeaec654318e4289", "feature": "F07.sync-first-during-run", "result": "xfail", "entry": "map run F07-session-socket.md: Sync first while a run holds the lock (`F07.sync-first-during-run`), known bug #6745.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F07-session-socket-01-sync-first-while-a-run-holds-the-lock-f0.json"], "note": "known bug reproduced (expected failure); failed at line 6: control-agent-server state cat \"evidence/F07.sync-first-during-run/order-$RUNC.json\" --check frames.2.kind eq sync"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f07-sync-first-during-run-2-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f07-sync-first-during-run-2-ledger.jsonl new file mode 100644 index 00000000000..2dcd7aba088 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f07-sync-first-during-run-2-ledger.jsonl @@ -0,0 +1 @@ +{"at": "2026-11-11T09:13:04Z", "run": "20261111-091219-fa68-map-f07", "checkout_sha": "edff6711505acb0f88d90dbbaeaec654318e4289", "feature": "F07.sync-first-during-run", "result": "xfail", "entry": "map run F07-session-socket.md: Sync first while a run holds the lock (`F07.sync-first-during-run`), known bug #6745.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F07-session-socket-01-sync-first-while-a-run-holds-the-lock-f0.json"], "note": "known bug reproduced (expected failure); failed at line 6: control-agent-server state cat \"evidence/F07.sync-first-during-run/order-$RUNC.json\" --check frames.2.kind eq sync"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f09-agent-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f09-agent-ledger.jsonl new file mode 100644 index 00000000000..c06bd006400 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f09-agent-ledger.jsonl @@ -0,0 +1,37 @@ +{"at": "2026-11-11T08:35:27Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.auth-required", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Key required (`F09.auth-required`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-01-key-required-f09-auth-required.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:35:29Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch to a caller-supplied LLM (`F09.switch-llm`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-02-switch-to-a-caller-supplied-llm-f09-swit.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:35:33Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm-ws", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Socket view of a switch (`F09.switch-llm-ws`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-03-socket-view-of-a-switch-f09-switch-llm-w.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:35:35Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm-errors", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch errors (`F09.switch-llm-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-04-switch-errors-f09-switch-llm-errors.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:35:41Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm-encrypted-key", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Mid-conversation switch with a cipher token (`F09.switch-llm-encrypted-key`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-05-mid-conversation-switch-with-a-cipher-to.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:35:50Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm-provider-connection", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch through a provider connection (`F09.switch-llm-provider-connection`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-06-switch-through-a-provider-connection-f09.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:35:53Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-profile", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch to a saved profile (`F09.switch-profile`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-07-switch-to-a-saved-profile-f09-switch-pro.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:35:56Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-profile-errors", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Profile switch errors (`F09.switch-profile-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-08-profile-switch-errors-f09-switch-profile.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:35:59Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-profile-edited", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: Switch to an edited profile (`F09.switch-profile-edited`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-09-switch-to-an-edited-profile-f09-switch-p.json"], "note": "known bug reproduced (expected failure); failed at line 9: control-agent-server api GET \"/api/conversations/$EID\" --quiet --check agent.llm.model eq openai/qa-f09-edit-2 --save F09.switch-profile-edited/get"} +{"at": "2026-11-11T08:36:03Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm-registered-usage-id", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch onto a registered usage id (`F09.switch-llm-registered-usage-id`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-10-switch-onto-a-registered-usage-id-f09-sw.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:36:28Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm-before-first-run", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: Switch before the first run (`F09.switch-llm-before-first-run`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-11-switch-before-the-first-run-f09-switch-l.json"], "note": "known bug reproduced (expected failure); failed at line 7: control-agent-server api GET \"/api/conversations/$BID\" --until-ok 20 --quiet --check stats.usage_to_metrics.qa-f09-early.accumulated_token_usage.prompt_tokens gt 0 --save F09.switch-llm-before-first-run/stats"} +{"at": "2026-11-11T08:36:53Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm-mid-run", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch during a running turn (`F09.switch-llm-mid-run`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-12-switch-during-a-running-turn-f09-switch-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:37:18Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm-mid-run-get", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: GET right after a mid-run switch (`F09.switch-llm-mid-run-get`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-13-get-right-after-a-mid-run-switch-f09-swi.json"], "note": "known bug reproduced (expected failure); failed at line 10: control-agent-server api GET \"/api/conversations/$GID\" --quiet --check agent.llm.usage_id eq qa-f09-midget --save F09.switch-llm-mid-run-get/get"} +{"at": "2026-11-11T08:37:19Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-model-native", "result": "pass", "entry": "map run F09-model-switching-plugins.md: ACP switch on a regular conversation (`F09.acp-model-native`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-14-acp-switch-on-a-regular-conversation-f09.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:37:22Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-model-deferred", "result": "pass", "entry": "map run F09-model-switching-plugins.md: ACP model before the first run (`F09.acp-model-deferred`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-15-acp-model-before-the-first-run-f09-acp-m.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:37:24Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-model-blank", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Blank ACP model (`F09.acp-model-blank`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-16-blank-acp-model-f09-acp-model-blank.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:37:27Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.switch-llm-acp-refused", "result": "pass", "entry": "map run F09-model-switching-plugins.md: LLM switches on an ACP conversation (`F09.switch-llm-acp-refused`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-17-llm-switches-on-an-acp-conversation-f09-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:37:53Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-model-live", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Live ACP switch (`F09.acp-model-live`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-18-live-acp-switch-f09-acp-model-live.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:37:55Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-model-live-unknown", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: Unknown model on a live session (`F09.acp-model-live-unknown`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-19-unknown-model-on-a-live-session-f09-acp-.json"], "note": "known bug reproduced (expected failure); failed at line 3: control-agent-server api POST \"/api/conversations/$LID/switch_acp_model\" --json '{\"model\": \"qa-f09-nope\"}' --expect 400,422 --save F09.acp-model-live-unknown/switch"} +{"at": "2026-11-11T08:38:02Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-model-live-timeout", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Live switch that times out (`F09.acp-model-live-timeout`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-20-live-switch-that-times-out-f09-acp-model.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:04Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-effort-native", "result": "pass", "entry": "map run F09-model-switching-plugins.md: ACP effort on a regular conversation (`F09.acp-effort-native`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-21-acp-effort-on-a-regular-conversation-f09.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:07Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-effort-deferred", "result": "pass", "entry": "map run F09-model-switching-plugins.md: ACP effort before the first run (`F09.acp-effort-deferred`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-22-acp-effort-before-the-first-run-f09-acp-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:13Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-model-id-bare", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Bare model id with a legacy effort suffix (`F09.acp-model-id-bare`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-23-bare-model-id-with-a-legacy-effort-suffi.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:17Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-effort-live", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Live effort switch (`F09.acp-effort-live`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-24-live-effort-switch-f09-acp-effort-live.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:19Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-effort-unoffered", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Unoffered effort on a live session (`F09.acp-effort-unoffered`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-25-unoffered-effort-on-a-live-session-f09-a.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:21Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-effort-model-switch", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Effort across a model switch (`F09.acp-effort-model-switch`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-26-effort-across-a-model-switch-f09-acp-eff.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:23Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-effort-live-timeout", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Live effort switch that times out (`F09.acp-effort-live-timeout`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-27-live-effort-switch-that-times-out-f09-ac.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:23Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.acp-effort-codex", "result": "blocked", "entry": "map run F09-model-switching-plugins.md: Codex effort levels (`F09.acp-effort-codex`), blocked: needs a ChatGPT subscription login for codex-acp (`CODEX_AUTH_JSON`).", "expected": "prerequisite available", "actual": "blocked (see the bullet)", "artifacts": [], "note": "bullet is marked blocked in the map"} +{"at": "2026-11-11T08:38:28Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.ts-client", "result": "pass", "entry": "map run F09-model-switching-plugins.md: TypeScript client (`F09.ts-client`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-29-typescript-client-f09-ts-client.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:29Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.ts-client-acp-docs", "result": "pass", "entry": "map run F09-model-switching-plugins.md: TypeScript docstrings for a pre-run ACP switch (`F09.ts-client-acp-docs`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-30-typescript-docstrings-for-a-pre-run-acp-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:32Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.ts-client-acp-effort", "result": "pass", "entry": "map run F09-model-switching-plugins.md: TypeScript effort switch (`F09.ts-client-acp-effort`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-31-typescript-effort-switch-f09-ts-client-a.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:37Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.load-plugin", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Load a plugin (`F09.load-plugin`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-32-load-a-plugin-f09-load-plugin.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:41Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.load-plugin-hook-runs", "result": "pass", "entry": "map run F09-model-switching-plugins.md: The loaded hook runs (`F09.load-plugin-hook-runs`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-33-the-loaded-hook-runs-f09-load-plugin-hoo.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:48Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.load-plugin-errors", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Plugin load errors (`F09.load-plugin-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-34-plugin-load-errors-f09-load-plugin-error.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:50Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.load-plugin-twice", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: Load the same plugin twice (`F09.load-plugin-twice`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-35-load-the-same-plugin-twice-f09-load-plug.json"], "note": "known bug reproduced (expected failure); failed at line 4: control-agent-server api GET \"/api/conversations/$QID\" --quiet --check hook_config.user_prompt_submit len-eq 1 --save F09.load-plugin-twice/get"} +{"at": "2026-11-11T08:39:05Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.load-plugin-sdk", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Load a plugin from the Python SDK (`F09.load-plugin-sdk`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-36-load-a-plugin-from-the-python-sdk-f09-lo.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:39:20Z", "run": "20261111-083508-5969-map-f09", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F09.restart-persist", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Restart (`F09.restart-persist`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-37-restart-f09-restart-persist.json"], "note": "automated map run; prose-only expectations were not re-judged"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f09-recheck-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f09-recheck-ledger.jsonl new file mode 100644 index 00000000000..95da209230d --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f09-recheck-ledger.jsonl @@ -0,0 +1,37 @@ +{"at": "2026-11-11T08:41:12Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.auth-required", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Key required (`F09.auth-required`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-01-key-required-f09-auth-required.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:13Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch to a caller-supplied LLM (`F09.switch-llm`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-02-switch-to-a-caller-supplied-llm-f09-swit.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:17Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm-ws", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Socket view of a switch (`F09.switch-llm-ws`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-03-socket-view-of-a-switch-f09-switch-llm-w.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:20Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm-errors", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch errors (`F09.switch-llm-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-04-switch-errors-f09-switch-llm-errors.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:27Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm-encrypted-key", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Mid-conversation switch with a cipher token (`F09.switch-llm-encrypted-key`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-05-mid-conversation-switch-with-a-cipher-to.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:35Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm-provider-connection", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch through a provider connection (`F09.switch-llm-provider-connection`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-06-switch-through-a-provider-connection-f09.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:40Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-profile", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch to a saved profile (`F09.switch-profile`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-07-switch-to-a-saved-profile-f09-switch-pro.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:43Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-profile-errors", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Profile switch errors (`F09.switch-profile-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-08-profile-switch-errors-f09-switch-profile.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:46Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-profile-edited", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: Switch to an edited profile (`F09.switch-profile-edited`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-09-switch-to-an-edited-profile-f09-switch-p.json"], "note": "known bug reproduced (expected failure); failed at line 9: control-agent-server api GET \"/api/conversations/$EID\" --quiet --check agent.llm.model eq openai/qa-f09-edit-2 --save F09.switch-profile-edited/get"} +{"at": "2026-11-11T08:41:50Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm-registered-usage-id", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch onto a registered usage id (`F09.switch-llm-registered-usage-id`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-10-switch-onto-a-registered-usage-id-f09-sw.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:13Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm-before-first-run", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: Switch before the first run (`F09.switch-llm-before-first-run`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-11-switch-before-the-first-run-f09-switch-l.json"], "note": "known bug reproduced (expected failure); failed at line 7: control-agent-server api GET \"/api/conversations/$BID\" --until-ok 20 --quiet --check stats.usage_to_metrics.qa-f09-early.accumulated_token_usage.prompt_tokens gt 0 --save F09.switch-llm-before-first-run/stats"} +{"at": "2026-11-11T08:42:38Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm-mid-run", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Switch during a running turn (`F09.switch-llm-mid-run`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-12-switch-during-a-running-turn-f09-switch-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:04Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm-mid-run-get", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: GET right after a mid-run switch (`F09.switch-llm-mid-run-get`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-13-get-right-after-a-mid-run-switch-f09-swi.json"], "note": "known bug reproduced (expected failure); failed at line 10: control-agent-server api GET \"/api/conversations/$GID\" --quiet --check agent.llm.usage_id eq qa-f09-midget --save F09.switch-llm-mid-run-get/get"} +{"at": "2026-11-11T08:43:06Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-model-native", "result": "pass", "entry": "map run F09-model-switching-plugins.md: ACP switch on a regular conversation (`F09.acp-model-native`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-14-acp-switch-on-a-regular-conversation-f09.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:08Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-model-deferred", "result": "pass", "entry": "map run F09-model-switching-plugins.md: ACP model before the first run (`F09.acp-model-deferred`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-15-acp-model-before-the-first-run-f09-acp-m.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:10Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-model-blank", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Blank ACP model (`F09.acp-model-blank`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-16-blank-acp-model-f09-acp-model-blank.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:13Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.switch-llm-acp-refused", "result": "pass", "entry": "map run F09-model-switching-plugins.md: LLM switches on an ACP conversation (`F09.switch-llm-acp-refused`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-17-llm-switches-on-an-acp-conversation-f09-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:39Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-model-live", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Live ACP switch (`F09.acp-model-live`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-18-live-acp-switch-f09-acp-model-live.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:41Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-model-live-unknown", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: Unknown model on a live session (`F09.acp-model-live-unknown`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-19-unknown-model-on-a-live-session-f09-acp-.json"], "note": "known bug reproduced (expected failure); failed at line 3: control-agent-server api POST \"/api/conversations/$LID/switch_acp_model\" --json '{\"model\": \"qa-f09-nope\"}' --expect 400,422 --save F09.acp-model-live-unknown/switch"} +{"at": "2026-11-11T08:43:48Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-model-live-timeout", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Live switch that times out (`F09.acp-model-live-timeout`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-20-live-switch-that-times-out-f09-acp-model.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:51Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-effort-native", "result": "pass", "entry": "map run F09-model-switching-plugins.md: ACP effort on a regular conversation (`F09.acp-effort-native`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-21-acp-effort-on-a-regular-conversation-f09.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:57Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-effort-deferred", "result": "pass", "entry": "map run F09-model-switching-plugins.md: ACP effort before the first run (`F09.acp-effort-deferred`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-22-acp-effort-before-the-first-run-f09-acp-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:05Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-model-id-bare", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Bare model id with a legacy effort suffix (`F09.acp-model-id-bare`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-23-bare-model-id-with-a-legacy-effort-suffi.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:10Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-effort-live", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Live effort switch (`F09.acp-effort-live`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-24-live-effort-switch-f09-acp-effort-live.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:12Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-effort-unoffered", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Unoffered effort on a live session (`F09.acp-effort-unoffered`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-25-unoffered-effort-on-a-live-session-f09-a.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:15Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-effort-model-switch", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Effort across a model switch (`F09.acp-effort-model-switch`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-26-effort-across-a-model-switch-f09-acp-eff.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:17Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-effort-live-timeout", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Live effort switch that times out (`F09.acp-effort-live-timeout`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-27-live-effort-switch-that-times-out-f09-ac.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:17Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.acp-effort-codex", "result": "blocked", "entry": "map run F09-model-switching-plugins.md: Codex effort levels (`F09.acp-effort-codex`), blocked: needs a ChatGPT subscription login for codex-acp (`CODEX_AUTH_JSON`).", "expected": "prerequisite available", "actual": "blocked (see the bullet)", "artifacts": [], "note": "bullet is marked blocked in the map"} +{"at": "2026-11-11T08:44:21Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.ts-client", "result": "pass", "entry": "map run F09-model-switching-plugins.md: TypeScript client (`F09.ts-client`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-29-typescript-client-f09-ts-client.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:22Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.ts-client-acp-docs", "result": "pass", "entry": "map run F09-model-switching-plugins.md: TypeScript docstrings for a pre-run ACP switch (`F09.ts-client-acp-docs`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-30-typescript-docstrings-for-a-pre-run-acp-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:24Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.ts-client-acp-effort", "result": "pass", "entry": "map run F09-model-switching-plugins.md: TypeScript effort switch (`F09.ts-client-acp-effort`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-31-typescript-effort-switch-f09-ts-client-a.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:29Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.load-plugin", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Load a plugin (`F09.load-plugin`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-32-load-a-plugin-f09-load-plugin.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:32Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.load-plugin-hook-runs", "result": "pass", "entry": "map run F09-model-switching-plugins.md: The loaded hook runs (`F09.load-plugin-hook-runs`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-33-the-loaded-hook-runs-f09-load-plugin-hoo.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:40Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.load-plugin-errors", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Plugin load errors (`F09.load-plugin-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-34-plugin-load-errors-f09-load-plugin-error.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:41Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.load-plugin-twice", "result": "xfail", "entry": "map run F09-model-switching-plugins.md: Load the same plugin twice (`F09.load-plugin-twice`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F09-model-switching-plugins-35-load-the-same-plugin-twice-f09-load-plug.json"], "note": "known bug reproduced (expected failure); failed at line 4: control-agent-server api GET \"/api/conversations/$QID\" --quiet --check hook_config.user_prompt_submit len-eq 1 --save F09.load-plugin-twice/get"} +{"at": "2026-11-11T08:44:55Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.load-plugin-sdk", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Load a plugin from the Python SDK (`F09.load-plugin-sdk`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-36-load-a-plugin-from-the-python-sdk-f09-lo.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:10Z", "run": "20261111-084052-4ce1-map-f09", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F09.restart-persist", "result": "pass", "entry": "map run F09-model-switching-plugins.md: Restart (`F09.restart-persist`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F09-model-switching-plugins-37-restart-f09-restart-persist.json"], "note": "automated map run; prose-only expectations were not re-judged"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f15-agent-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f15-agent-ledger.jsonl new file mode 100644 index 00000000000..7101b4c25e8 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f15-agent-ledger.jsonl @@ -0,0 +1,36 @@ +{"at": "2026-11-11T08:44:12Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.auth", "result": "pass", "entry": "map run F15-git.md: Session key required (`F15.auth`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-01-session-key-required-f15-auth.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:13Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.changes", "result": "pass", "entry": "map run F15-git.md: Working-tree changes (`F15.changes`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-02-working-tree-changes-f15-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:14Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.changes-statuses", "result": "pass", "entry": "map run F15-git.md: Staged deletions, additions and renames (`F15.changes-statuses`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-03-staged-deletions-additions-and-renames-f.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:17Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.changes-base", "result": "pass", "entry": "map run F15-git.md: Default display base (`F15.changes-base`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-04-default-display-base-f15-changes-base.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:19Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.changes-ref", "result": "pass", "entry": "map run F15-git.md: Explicit base and its errors (`F15.changes-ref`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-05-explicit-base-and-its-errors-f15-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:20Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.changes-nested-repo", "result": "pass", "entry": "map run F15-git.md: Nested repository (`F15.changes-nested-repo`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-06-nested-repository-f15-changes-nested-rep.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:21Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.changes-repo-below-path", "result": "pass", "entry": "map run F15-git.md: Repository below the requested directory (`F15.changes-repo-below-path`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-07-repository-below-the-requested-directory.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:23Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.changes-special-names", "result": "pass", "entry": "map run F15-git.md: File names with a space or non-ASCII characters (`F15.changes-special-names`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-08-file-names-with-a-space-or-non-ascii-cha.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:24Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.changes-subdir-paths", "result": "pass", "entry": "map run F15-git.md: Subdirectory of a repository (`F15.changes-subdir-paths`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-09-subdirectory-of-a-repository-f15-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:28Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.non-repo", "result": "pass", "entry": "map run F15-git.md: Not a repository, or no commits yet (`F15.non-repo`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-10-not-a-repository-or-no-commits-yet-f15-n.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:30Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.diff", "result": "pass", "entry": "map run F15-git.md: Per-file diff (`F15.diff`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-11-per-file-diff-f15-diff.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:32Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.diff-commit", "result": "pass", "entry": "map run F15-git.md: Diff of one commit (`F15.diff-commit`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-12-diff-of-one-commit-f15-diff-commit.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:35Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.diff-errors", "result": "pass", "entry": "map run F15-git.md: Diff errors (`F15.diff-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-13-diff-errors-f15-diff-errors.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:38Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.diff-whitespace", "result": "pass", "entry": "map run F15-git.md: Unchanged-file whitespace (`F15.diff-whitespace`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-14-unchanged-file-whitespace-f15-diff-white.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:39Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.diff-deleted-file", "result": "pass", "entry": "map run F15-git.md: Diff of a deleted file (`F15.diff-deleted-file`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-15-diff-of-a-deleted-file-f15-diff-deleted-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:42Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.commits", "result": "pass", "entry": "map run F15-git.md: Commit history (`F15.commits`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-16-commit-history-f15-commits.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:46Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.commit-changes", "result": "pass", "entry": "map run F15-git.md: Files of one commit (`F15.commit-changes`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-17-files-of-one-commit-f15-commit-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:49Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.scoped", "result": "pass", "entry": "map run F15-git.md: Conversation-scoped views (`F15.scoped`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-18-conversation-scoped-views-f15-scoped.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:58Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.scoped-guard", "result": "pass", "entry": "map run F15-git.md: Workspace guard (`F15.scoped-guard`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-19-workspace-guard-f15-scoped-guard.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:12Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.scoped-restart", "result": "pass", "entry": "map run F15-git.md: After a restart (`F15.scoped-restart`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-20-after-a-restart-f15-scoped-restart.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:20Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.sdk-workspace", "result": "pass", "entry": "map run F15-git.md: Python SDK workspace (`F15.sdk-workspace`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-21-python-sdk-workspace-f15-sdk-workspace.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:22Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.ts-client", "result": "pass", "entry": "map run F15-git.md: TypeScript client (`F15.ts-client`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-22-typescript-client-f15-ts-client.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:30Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.agent-changes", "result": "pass", "entry": "map run F15-git.md: What the agent changed (`F15.agent-changes`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-23-what-the-agent-changed-f15-agent-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:34Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.repo-search-token", "result": "pass", "entry": "map run F15-git.md: Token lookup (`F15.repo-search-token`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-24-token-lookup-f15-repo-search-token.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:37Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.repo-search-errors", "result": "pass", "entry": "map run F15-git.md: Provider and parameter validation (`F15.repo-search-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-25-provider-and-parameter-validation-f15-re.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:46:00Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.repo-search-unreachable", "result": "pass", "entry": "map run F15-git.md: GitHub unreachable (`F15.repo-search-unreachable`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-26-github-unreachable-f15-repo-search-unrea.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:46:37Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.repo-search-timeout", "result": "pass", "entry": "map run F15-git.md: GitHub timeout (`F15.repo-search-timeout`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-27-github-timeout-f15-repo-search-timeout.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:46:37Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.repo-search-github", "result": "blocked", "entry": "map run F15-git.md: Live GitHub listing (`F15.repo-search-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`.", "expected": "prerequisite available", "actual": "blocked (see the bullet)", "artifacts": [], "note": "bullet is marked blocked in the map"} +{"at": "2026-11-11T08:47:03Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.repo-search-rejected-token", "result": "pass", "entry": "map run F15-git.md: Rejected provider token (`F15.repo-search-rejected-token`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-29-rejected-provider-token-f15-repo-search-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:47:04Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.pr-status-token", "result": "pass", "entry": "map run F15-git.md: Pull-request status without a token (`F15.pr-status-token`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-30-pull-request-status-without-a-token-f15-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:47:10Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.pr-status-errors", "result": "pass", "entry": "map run F15-git.md: Pull-request status validation (`F15.pr-status-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-31-pull-request-status-validation-f15-pr-st.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:47:31Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.pr-status-unreachable", "result": "pass", "entry": "map run F15-git.md: GitHub unreachable for pull-request status (`F15.pr-status-unreachable`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-32-github-unreachable-for-pull-request-stat.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:47:48Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.pr-status-state", "result": "pass", "entry": "map run F15-git.md: Pull-request state and CI result (`F15.pr-status-state`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-33-pull-request-state-and-ci-result-f15-pr-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:47:52Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.pr-status-provider-errors", "result": "pass", "entry": "map run F15-git.md: Provider errors for pull-request status (`F15.pr-status-provider-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-34-provider-errors-for-pull-request-status-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:48:20Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.pr-status-timeout", "result": "pass", "entry": "map run F15-git.md: Pull-request status timeout (`F15.pr-status-timeout`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-35-pull-request-status-timeout-f15-pr-statu.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:48:20Z", "run": "20261111-084346-c245-map-f15", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F15.pr-status-github", "result": "blocked", "entry": "map run F15-git.md: Live pull-request status (`F15.pr-status-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`.", "expected": "prerequisite available", "actual": "blocked (see the bullet)", "artifacts": [], "note": "bullet is marked blocked in the map"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f15-recheck-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f15-recheck-ledger.jsonl new file mode 100644 index 00000000000..7f22966e258 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f15-recheck-ledger.jsonl @@ -0,0 +1,36 @@ +{"at": "2026-11-11T08:54:44Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.auth", "result": "pass", "entry": "map run F15-git.md: Session key required (`F15.auth`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-01-session-key-required-f15-auth.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:54:45Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.changes", "result": "pass", "entry": "map run F15-git.md: Working-tree changes (`F15.changes`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-02-working-tree-changes-f15-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:54:45Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.changes-statuses", "result": "pass", "entry": "map run F15-git.md: Staged deletions, additions and renames (`F15.changes-statuses`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-03-staged-deletions-additions-and-renames-f.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:54:47Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.changes-base", "result": "pass", "entry": "map run F15-git.md: Default display base (`F15.changes-base`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-04-default-display-base-f15-changes-base.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:54:50Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.changes-ref", "result": "pass", "entry": "map run F15-git.md: Explicit base and its errors (`F15.changes-ref`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-05-explicit-base-and-its-errors-f15-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:54:51Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.changes-nested-repo", "result": "pass", "entry": "map run F15-git.md: Nested repository (`F15.changes-nested-repo`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-06-nested-repository-f15-changes-nested-rep.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:54:52Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.changes-repo-below-path", "result": "pass", "entry": "map run F15-git.md: Repository below the requested directory (`F15.changes-repo-below-path`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-07-repository-below-the-requested-directory.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:54:53Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.changes-special-names", "result": "pass", "entry": "map run F15-git.md: File names with a space or non-ASCII characters (`F15.changes-special-names`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-08-file-names-with-a-space-or-non-ascii-cha.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:54:55Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.changes-subdir-paths", "result": "pass", "entry": "map run F15-git.md: Subdirectory of a repository (`F15.changes-subdir-paths`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-09-subdirectory-of-a-repository-f15-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:54:59Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.non-repo", "result": "pass", "entry": "map run F15-git.md: Not a repository, or no commits yet (`F15.non-repo`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-10-not-a-repository-or-no-commits-yet-f15-n.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:03Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.diff", "result": "pass", "entry": "map run F15-git.md: Per-file diff (`F15.diff`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-11-per-file-diff-f15-diff.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:06Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.diff-commit", "result": "pass", "entry": "map run F15-git.md: Diff of one commit (`F15.diff-commit`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-12-diff-of-one-commit-f15-diff-commit.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:11Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.diff-errors", "result": "pass", "entry": "map run F15-git.md: Diff errors (`F15.diff-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-13-diff-errors-f15-diff-errors.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:16Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.diff-whitespace", "result": "pass", "entry": "map run F15-git.md: Unchanged-file whitespace (`F15.diff-whitespace`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-14-unchanged-file-whitespace-f15-diff-white.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:18Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.diff-deleted-file", "result": "pass", "entry": "map run F15-git.md: Diff of a deleted file (`F15.diff-deleted-file`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-15-diff-of-a-deleted-file-f15-diff-deleted-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:21Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.commits", "result": "pass", "entry": "map run F15-git.md: Commit history (`F15.commits`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-16-commit-history-f15-commits.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:25Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.commit-changes", "result": "pass", "entry": "map run F15-git.md: Files of one commit (`F15.commit-changes`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-17-files-of-one-commit-f15-commit-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:28Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.scoped", "result": "pass", "entry": "map run F15-git.md: Conversation-scoped views (`F15.scoped`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-18-conversation-scoped-views-f15-scoped.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:37Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.scoped-guard", "result": "pass", "entry": "map run F15-git.md: Workspace guard (`F15.scoped-guard`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-19-workspace-guard-f15-scoped-guard.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:55:54Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.scoped-restart", "result": "pass", "entry": "map run F15-git.md: After a restart (`F15.scoped-restart`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-20-after-a-restart-f15-scoped-restart.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:56:02Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.sdk-workspace", "result": "pass", "entry": "map run F15-git.md: Python SDK workspace (`F15.sdk-workspace`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-21-python-sdk-workspace-f15-sdk-workspace.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:56:04Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.ts-client", "result": "pass", "entry": "map run F15-git.md: TypeScript client (`F15.ts-client`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-22-typescript-client-f15-ts-client.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:56:13Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.agent-changes", "result": "pass", "entry": "map run F15-git.md: What the agent changed (`F15.agent-changes`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-23-what-the-agent-changed-f15-agent-changes.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:56:18Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.repo-search-token", "result": "pass", "entry": "map run F15-git.md: Token lookup (`F15.repo-search-token`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-24-token-lookup-f15-repo-search-token.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:56:21Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.repo-search-errors", "result": "pass", "entry": "map run F15-git.md: Provider and parameter validation (`F15.repo-search-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-25-provider-and-parameter-validation-f15-re.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:56:43Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.repo-search-unreachable", "result": "pass", "entry": "map run F15-git.md: GitHub unreachable (`F15.repo-search-unreachable`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-26-github-unreachable-f15-repo-search-unrea.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:57:22Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.repo-search-timeout", "result": "pass", "entry": "map run F15-git.md: GitHub timeout (`F15.repo-search-timeout`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-27-github-timeout-f15-repo-search-timeout.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:57:22Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.repo-search-github", "result": "blocked", "entry": "map run F15-git.md: Live GitHub listing (`F15.repo-search-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`.", "expected": "prerequisite available", "actual": "blocked (see the bullet)", "artifacts": [], "note": "bullet is marked blocked in the map"} +{"at": "2026-11-11T08:57:48Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.repo-search-rejected-token", "result": "pass", "entry": "map run F15-git.md: Rejected provider token (`F15.repo-search-rejected-token`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-29-rejected-provider-token-f15-repo-search-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:57:49Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.pr-status-token", "result": "pass", "entry": "map run F15-git.md: Pull-request status without a token (`F15.pr-status-token`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-30-pull-request-status-without-a-token-f15-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:57:59Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.pr-status-errors", "result": "pass", "entry": "map run F15-git.md: Pull-request status validation (`F15.pr-status-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-31-pull-request-status-validation-f15-pr-st.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:58:39Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.pr-status-unreachable", "result": "pass", "entry": "map run F15-git.md: GitHub unreachable for pull-request status (`F15.pr-status-unreachable`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-32-github-unreachable-for-pull-request-stat.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:59:09Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.pr-status-state", "result": "pass", "entry": "map run F15-git.md: Pull-request state and CI result (`F15.pr-status-state`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-33-pull-request-state-and-ci-result-f15-pr-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:59:16Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.pr-status-provider-errors", "result": "pass", "entry": "map run F15-git.md: Provider errors for pull-request status (`F15.pr-status-provider-errors`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-34-provider-errors-for-pull-request-status-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:59:53Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.pr-status-timeout", "result": "pass", "entry": "map run F15-git.md: Pull-request status timeout (`F15.pr-status-timeout`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F15-git-35-pull-request-status-timeout-f15-pr-statu.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:59:53Z", "run": "20261111-085423-8ec2-map-f15", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F15.pr-status-github", "result": "blocked", "entry": "map run F15-git.md: Live pull-request status (`F15.pr-status-github`), blocked: needs `QA_GITHUB_TOKEN` and egress to `api.github.com`.", "expected": "prerequisite available", "actual": "blocked (see the bullet)", "artifacts": [], "note": "bullet is marked blocked in the map"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f21-full-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f21-full-ledger.jsonl new file mode 100644 index 00000000000..7b4f7afc100 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f21-full-ledger.jsonl @@ -0,0 +1,32 @@ +{"at": "2026-11-11T08:37:41Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.auth-required", "result": "pass", "entry": "map run F21-agent-profiles.md: No key, no access (`F21.auth-required`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-01-no-key-no-access-f21-auth-required.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:37:44Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.lazy-seed", "result": "pass", "entry": "map run F21-agent-profiles.md: Lazy seed (`F21.lazy-seed`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-02-lazy-seed-f21-lazy-seed.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:01Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.seed-no-ghost-llm", "result": "pass", "entry": "map run F21-agent-profiles.md: No ghost LLM profile (`F21.seed-no-ghost-llm`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-03-no-ghost-llm-profile-f21-seed-no-ghost-l.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:12Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.seed-acp", "result": "pass", "entry": "map run F21-agent-profiles.md: ACP seed (`F21.seed-acp`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-04-acp-seed-f21-seed-acp.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:14Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.save-create", "result": "pass", "entry": "map run F21-agent-profiles.md: Create (`F21.save-create`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-05-create-f21-save-create.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:15Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.save-overwrite", "result": "pass", "entry": "map run F21-agent-profiles.md: Overwrite (`F21.save-overwrite`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-06-overwrite-f21-save-overwrite.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:16Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.acp-kind", "result": "pass", "entry": "map run F21-agent-profiles.md: ACP profile (`F21.acp-kind`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-07-acp-profile-f21-acp-kind.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:21Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.save-validation", "result": "pass", "entry": "map run F21-agent-profiles.md: Save validation (`F21.save-validation`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-08-save-validation-f21-save-validation.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:29Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.name-rules", "result": "pass", "entry": "map run F21-agent-profiles.md: Name rules (`F21.name-rules`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-09-name-rules-f21-name-rules.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:34Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.not-found", "result": "pass", "entry": "map run F21-agent-profiles.md: Unknown names and ids (`F21.not-found`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-10-unknown-names-and-ids-f21-not-found.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:36Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.activate", "result": "pass", "entry": "map run F21-agent-profiles.md: Activate by id (`F21.activate`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-11-activate-by-id-f21-activate.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:38Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.activate-corrupt-settings", "result": "pass", "entry": "map run F21-agent-profiles.md: Activate over a corrupt settings file (`F21.activate-corrupt-settings`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-12-activate-over-a-corrupt-settings-file-f2.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:43Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.rename", "result": "pass", "entry": "map run F21-agent-profiles.md: Rename (`F21.rename`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-13-rename-f21-rename.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:49Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.delete", "result": "pass", "entry": "map run F21-agent-profiles.md: Delete (`F21.delete`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-14-delete-f21-delete.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:38:55Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.delete-active-acp-reset", "result": "pass", "entry": "map run F21-agent-profiles.md: Active ACP reset (`F21.delete-active-acp-reset`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-15-active-acp-reset-f21-delete-active-acp-r.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:39:02Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.materialize-valid", "result": "pass", "entry": "map run F21-agent-profiles.md: Materialize a resolvable profile (`F21.materialize-valid`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-16-materialize-a-resolvable-profile-f21-mat.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:39:04Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.materialize-dangling", "result": "pass", "entry": "map run F21-agent-profiles.md: Dangling references (`F21.materialize-dangling`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-17-dangling-references-f21-materialize-dang.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:39:09Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.materialize-draft", "result": "pass", "entry": "map run F21-agent-profiles.md: Draft preview (`F21.materialize-draft`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-18-draft-preview-f21-materialize-draft.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:39:54Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.limit", "result": "pass", "entry": "map run F21-agent-profiles.md: At most 50 (`F21.limit`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-19-at-most-50-f21-limit.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:39:57Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.corrupt-file", "result": "pass", "entry": "map run F21-agent-profiles.md: Corrupt file (`F21.corrupt-file`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-20-corrupt-file-f21-corrupt-file.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:40:09Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.conversation-start", "result": "pass", "entry": "map run F21-agent-profiles.md: Start a conversation from a profile (`F21.conversation-start`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-21-start-a-conversation-from-a-profile-f21-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:40:15Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.secret-scope", "result": "pass", "entry": "map run F21-agent-profiles.md: Secret scope (`F21.secret-scope`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-22-secret-scope-f21-secret-scope.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:40:20Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.launch-additions", "result": "pass", "entry": "map run F21-agent-profiles.md: Launch additions (`F21.launch-additions`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-23-launch-additions-f21-launch-additions.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:40:23Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.ts-client", "result": "pass", "entry": "map run F21-agent-profiles.md: TypeScript client (`F21.ts-client`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-24-typescript-client-f21-ts-client.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:40:38Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.restart", "result": "pass", "entry": "map run F21-agent-profiles.md: Restart (`F21.restart`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-25-restart-f21-restart.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:07Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.acp-skill-sourcing", "result": "pass", "entry": "map run F21-agent-profiles.md: ACP skill sourcing (`F21.acp-skill-sourcing`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-26-acp-skill-sourcing-f21-acp-skill-sourcin.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:37Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.store-busy", "result": "pass", "entry": "map run F21-agent-profiles.md: Store busy (`F21.store-busy`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-27-store-busy-f21-store-busy.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:09Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.launch-store-busy", "result": "pass", "entry": "map run F21-agent-profiles.md: Busy store at conversation start (`F21.launch-store-busy`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-28-busy-store-at-conversation-start-f21-lau.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:15Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.busy-blocks-server", "result": "pass", "entry": "map run F21-agent-profiles.md: Lock wait does not stall the server (`F21.busy-blocks-server`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-29-lock-wait-does-not-stall-the-server-f21-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:18Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.dangling-llm-report", "result": "pass", "entry": "map run F21-agent-profiles.md: How a dangling LLM reference is reported (`F21.dangling-llm-report`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-30-how-a-dangling-llm-reference-is-reported.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:20Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.ts-payload-types", "result": "fail", "entry": "map run F21-agent-profiles.md: TypeScript payload types (`F21.ts-payload-types`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 2", "artifacts": ["evidence/map-run/F21-agent-profiles-31-typescript-payload-types-f21-ts-payload-.json"], "note": "automated map run; prose-only expectations were not re-judged; failed at line 25: $TSC \"$D/contract.ts\""} +{"at": "2026-11-11T08:42:24Z", "run": "20261111-083718-2007-map-f21", "checkout_sha": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "feature": "F21.json-suffix-identity", "result": "pass", "entry": "map run F21-agent-profiles.md: `.json` names keep their identity (`F21.json-suffix-identity`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F21-agent-profiles-32-json-names-keep-their-identity-f21-json-.json"], "note": "automated map run; prose-only expectations were not re-judged"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f24-offline-safe-rerun-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f24-offline-safe-rerun-ledger.jsonl new file mode 100644 index 00000000000..ce7ea049e6d --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f24-offline-safe-rerun-ledger.jsonl @@ -0,0 +1 @@ +{"at": "2026-11-11T08:43:29Z", "run": "20261111-084256-5c92-map-f24", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F24.offline-safe", "result": "pass", "entry": "map run F24-openai-subscription.md: Offline-safe routes (`F24.offline-safe`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F24-openai-subscription-01-offline-safe-routes-f24-offline-safe.json"], "note": "automated map run; prose-only expectations were not re-judged"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f27-full-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f27-full-ledger.jsonl new file mode 100644 index 00000000000..74b989c989c --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f27-full-ledger.jsonl @@ -0,0 +1,32 @@ +{"at": "2026-11-11T08:41:40Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.auth", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Session key required (`F27.auth`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-01-session-key-required-f27-auth.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:42Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.validation", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Bad bodies (`F27.validation`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-02-bad-bodies-f27-validation.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:43Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.tools-list", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Registered tools (`F27.tools-list`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-03-registered-tools-f27-tools-list.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:44Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.tools-redirect", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Slash redirects (`F27.tools-redirect`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-04-slash-redirects-f27-tools-redirect.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:44Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.catalog", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Tool catalog (`F27.catalog`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-05-tool-catalog-f27-catalog.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:53Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.child-conversation-tool", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Child conversation tool (`F27.child-conversation-tool`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-06-child-conversation-tool-f27-child-conver.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:54Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.hooks-project", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Project hooks (`F27.hooks-project`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-07-project-hooks-f27-hooks-project.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:54Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.hooks-formats", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Formats (`F27.hooks-formats`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-08-formats-f27-hooks-formats.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:41:57Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.hooks-absent", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: No hooks (`F27.hooks-absent`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-09-no-hooks-f27-hooks-absent.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:00Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.hooks-invalid", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Broken hooks.json (`F27.hooks-invalid`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-10-broken-hooks-json-f27-hooks-invalid.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:01Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.subagents-builtin", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Sub-agent built-ins (`F27.subagents-builtin`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-11-sub-agent-built-ins-f27-subagents-builti.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:02Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.subagents-project", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Project sub-agents (`F27.subagents-project`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-12-project-sub-agents-f27-subagents-project.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:02Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.subagents-user", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: User sub-agents (`F27.subagents-user`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-13-user-sub-agents-f27-subagents-user.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:03Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.subagents-precedence", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Precedence (`F27.subagents-precedence`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-14-precedence-f27-subagents-precedence.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:04Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.subagents-invalid", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Invalid definitions (`F27.subagents-invalid`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-15-invalid-definitions-f27-subagents-invali.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:07Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.subagents-bad-skill", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Agent with a missing skill (`F27.subagents-bad-skill`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-16-agent-with-a-missing-skill-f27-subagents.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:23Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.hooks-conversation", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Hooks in a conversation (`F27.hooks-conversation`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-17-hooks-in-a-conversation-f27-hooks-conver.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:29Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.hooks-not-automatic", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Hooks are not loaded implicitly (`F27.hooks-not-automatic`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-18-hooks-are-not-loaded-implicitly-f27-hook.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:41Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.hooks-block", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Blocking hooks (`F27.hooks-block`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-19-blocking-hooks-f27-hooks-block.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:42:42Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.catalog-sealed", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Client tools stay out of the catalog (`F27.catalog-sealed`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-20-client-tools-stay-out-of-the-catalog-f27.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:09Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.client-tool-call", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Client tool call (`F27.client-tool-call`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-21-client-tool-call-f27-client-tool-call.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:43:23Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.agent-definitions", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Forwarded sub-agent definitions (`F27.agent-definitions`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-22-forwarded-sub-agent-definitions-f27-agen.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:44:07Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.custom-tool-modules", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Custom tool module (`F27.custom-tool-modules`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-23-custom-tool-module-f27-custom-tool-modul.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:08Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.import-modules", "result": "fail", "entry": "map run F27-hooks-subagents-tools.md: Startup imports (`F27.import-modules`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-24-startup-imports-f27-import-modules.json"], "note": "automated map run; prose-only expectations were not re-judged; failed at line 13: curl -s --noproxy '*' --retry 60 --retry-connrefused --retry-delay 1 --retry-max-time 120 -o /dev/null \"http://127.0.0.1:$IPORT/alive\""} +{"at": "2026-11-11T08:45:20Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.custom-tool-missing-on-resume", "result": "fail", "entry": "map run F27-hooks-subagents-tools.md: Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-25-tool-module-gone-on-resume-f27-custom-to.json"], "note": "automated map run; prose-only expectations were not re-judged; failed at line 4: control-agent-server api POST \"/api/conversations/$MCID/events\" --json '{\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"Reply with the single word ok.\"}], \"run\": false}' --expect 2xx,4xx --save F27.custom-tool-missing-on-resume/send"} +{"at": "2026-11-11T08:45:20Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.browser-playwright-path", "result": "blocked", "entry": "map run F27-hooks-subagents-tools.md: Playwright browsers directory (`F27.browser-playwright-path`), blocked on a machine with Chromium at a standard path (needs Chromium only in a Playwright directory).", "expected": "prerequisite available", "actual": "blocked (see the bullet)", "artifacts": [], "note": "bullet is marked blocked in the map"} +{"at": "2026-11-11T08:45:20Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.browser-probe", "result": "blocked", "entry": "map run F27-hooks-subagents-tools.md: Browser probe (`F27.browser-probe`), blocked on a machine with Chromium at a standard path (needs Chromium only in a Playwright directory).", "expected": "prerequisite available", "actual": "blocked (see the bullet)", "artifacts": [], "note": "bullet is marked blocked in the map"} +{"at": "2026-11-11T08:45:41Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.browser-disabled-launch", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Default tools follow the browser setting (`F27.browser-disabled-launch`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-28-default-tools-follow-the-browser-setting.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:51Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.browser-disabled", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Browser disabled by config (`F27.browser-disabled`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-29-browser-disabled-by-config-f27-browser-d.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:45:52Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.server-info-browser-disabled", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Usable tools with the browser disabled (`F27.server-info-browser-disabled`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-30-usable-tools-with-the-browser-disabled-f.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:46:02Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.subagents-browser-disabled", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Browser-only sub-agent with the browser disabled (`F27.subagents-browser-disabled`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-31-browser-only-sub-agent-with-the-browser-.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T08:46:29Z", "run": "20261111-084114-3c93-map-f27", "checkout_sha": "faeba5ef369c23258b5538059337c5e9119c6c05", "feature": "F27.catalog-docker-runtime", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Docker conversation runtime (`F27.catalog-docker-runtime`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-32-docker-conversation-runtime-f27-catalog-.json"], "note": "automated map run; prose-only expectations were not re-judged"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f27-resume-1-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f27-resume-1-ledger.jsonl new file mode 100644 index 00000000000..86a1b8426c9 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f27-resume-1-ledger.jsonl @@ -0,0 +1,2 @@ +{"at": "2026-11-11T09:01:20Z", "run": "20261111-090008-47b9-map-f27", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F27.custom-tool-modules", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Custom tool module (`F27.custom-tool-modules`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-01-custom-tool-module-f27-custom-tool-modul.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T09:01:38Z", "run": "20261111-090008-47b9-map-f27", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F27.custom-tool-missing-on-resume", "result": "xfail", "entry": "map run F27-hooks-subagents-tools.md: Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-02-tool-module-gone-on-resume-f27-custom-to.json"], "note": "known bug reproduced (expected failure); failed at line 4: control-agent-server api POST \"/api/conversations/$MCID/events\" --json '{\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"Reply with the single word ok.\"}], \"run\": false}' --expect 2xx,4xx --save F27.custom-tool-missing-on-resume/send"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/f27-resume-2-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/f27-resume-2-ledger.jsonl new file mode 100644 index 00000000000..edac8113b46 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/f27-resume-2-ledger.jsonl @@ -0,0 +1,2 @@ +{"at": "2026-11-11T09:03:15Z", "run": "20261111-090141-d9c4-map-f27", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F27.custom-tool-modules", "result": "pass", "entry": "map run F27-hooks-subagents-tools.md: Custom tool module (`F27.custom-tool-modules`).", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 0", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-01-custom-tool-module-f27-custom-tool-modul.json"], "note": "automated map run; prose-only expectations were not re-judged"} +{"at": "2026-11-11T09:03:28Z", "run": "20261111-090141-d9c4-map-f27", "checkout_sha": "391b5546481d511261a4850e458b487478c663f0", "feature": "F27.custom-tool-missing-on-resume", "result": "xfail", "entry": "map run F27-hooks-subagents-tools.md: Tool module gone on resume (`F27.custom-tool-missing-on-resume`), known bug.", "expected": "every command in the bullet exits 0 (its --expect/--check assertions hold)", "actual": "exit 1", "artifacts": ["evidence/map-run/F27-hooks-subagents-tools-02-tool-module-gone-on-resume-f27-custom-to.json"], "note": "known bug reproduced (expected failure); failed at line 4: control-agent-server api POST \"/api/conversations/$MCID/events\" --json '{\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"Reply with the single word ok.\"}], \"run\": false}' --expect 2xx,4xx --save F27.custom-tool-missing-on-resume/send"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/s01-structured-output-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/s01-structured-output-ledger.jsonl new file mode 100644 index 00000000000..480b189df33 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/s01-structured-output-ledger.jsonl @@ -0,0 +1 @@ +{"feature": "S01.structured-output", "result": "pass", "proof": ["example", "live", "scripted"], "entry": "**Structured output (`S01.structured-output`).** `response_schema` on `TerminalTool` and on `FinishTool`; two invalid schemas. | **Examples 18, 20, 45 and 56 (`S01.message-while-running`, `S01.stuck-detect`, `S01.parallel-tools`, `S01.structured-output`).** Live; example 20 prints `Final stuck status: False` when the model refuses to repeat itself, so only its completion is checked.", "note": "bullet 18: pass; bullet 21: pass", "revision": "391b5546481d511261a4850e458b487478c663f0", "transcript": "/sdk/s01fix/r1/transcripts/S01-agent-loop-18.log", "at": "2026-11-11T09:03:26+00:00"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/s04-interactive-terminal-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/s04-interactive-terminal-ledger.jsonl new file mode 100644 index 00000000000..ffc7aa407bc --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/s04-interactive-terminal-ledger.jsonl @@ -0,0 +1 @@ +{"feature": "S04.interactive-terminal", "result": "fail", "proof": [], "entry": "**Interactive terminal (`S04.interactive-terminal`).** With `no_change_timeout_seconds=3`, `python3` waits for input after 3 s and the next commands are typed into the REPL; a callback collects every LLM message, tool results included. A program waiting on `input()` is then answered the way the terminal tool tells the model to: `is_input=True`. | **Example 06 and integration test t07 (`S04.interactive-terminal`).** Live.", "note": "bullet 19: fail (block 1 exited 1); bullet 25: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY))", "revision": "c707d3737e8d1b7bbc914bbb94f9161bb0593b4d", "transcript": "/sdk/S04-it-1/transcripts/S04-local-conversation-19.log", "at": "2026-11-11T09:18:02+00:00"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/s04-persist-events-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/s04-persist-events-ledger.jsonl new file mode 100644 index 00000000000..eda11f49650 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/s04-persist-events-ledger.jsonl @@ -0,0 +1,2 @@ +{"feature": "S04.persist-events", "result": "pass", "proof": ["scripted"], "entry": "**The on-disk layout, written event by event (`S04.persist-events`, `S04.resume`, `S04.callbacks`).** Through `control-sdk`, then from a program whose callback looks for each event's file as the event arrives. | **Damaged event files (`S04.persist-events`).** A finished conversation is reopened after one change at a time to its `events/` folder: an empty file (what an interrupted write leaves), invalid JSON, a deleted file. The conversation reopens every time without an error: a deleted file is skipped and the other events reload; an empty or invalid file is read as a `ConversationErrorEvent` stand-in (code `UnreadableEventFile`) in the same place, so the event count is unchanged (`S05.event-log-recovery`). | **Examples 09, 10, 11 and 12 (`S04.pause`, `S04.persist-events`, `S04.resume`, `S04.async-callback`, `S04.secrets-update`, `S04.secrets-masking`, `S04.visualizer-default`).** Live; example 10 also needs `uvx` and network.", "note": "bullet 2: pass; bullet 3: pass; bullet 23: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY))", "revision": "0da9cf850a0ead41633d0ca89a0c611cb1a83eff", "transcript": "/sdk/S04-persist2/transcripts/S04-local-conversation-02.log", "at": "2026-11-11T09:29:38+00:00"} +{"feature": "S05.event-log-recovery", "result": "pass", "proof": ["scripted"], "entry": "**Damaged event files (`S04.persist-events`).** A finished conversation is reopened after one change at a time to its `events/` folder: an empty file (what an interrupted write leaves), invalid JSON, a deleted file. The conversation reopens every time without an error: a deleted file is skipped and the other events reload; an empty or invalid file is read as a `ConversationErrorEvent` stand-in (code `UnreadableEventFile`) in the same place, so the event count is unchanged (`S05.event-log-recovery`).", "note": "bullet 3: pass", "revision": "0da9cf850a0ead41633d0ca89a0c611cb1a83eff", "transcript": "/sdk/S04-persist2/transcripts/S04-local-conversation-03.log", "at": "2026-11-11T09:29:38+00:00"} diff --git a/.pr/map-maintenance/2026-11-11/evidence/s21-effort-ledger.jsonl b/.pr/map-maintenance/2026-11-11/evidence/s21-effort-ledger.jsonl new file mode 100644 index 00000000000..b489c68941c --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/evidence/s21-effort-ledger.jsonl @@ -0,0 +1,26 @@ +{"feature": "S21.construct", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment.", "note": "bullet 1: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.step", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment. | **Example 40 with Claude Code (`S21.live-turn`, `S21.step`, `S21.ask-agent`, `S21.usage-cost`), blocked: needs `ANTHROPIC_API_KEY` (optionally `ANTHROPIC_BASE_URL` for a LiteLLM proxy), plus `npx` and network for the image URL.** Run it from the repo root. The agent lists the `openhands-sdk/openhands/sdk/agent/` files, describes the python.org image and answers the `ask_agent` question. The example prints `EXAMPLE_COST` twice, then `Done!`.", "note": "bullet 1: pass; bullet 14: blocked (bullet is labeled blocked)", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.tool-call-events", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment.", "note": "bullet 1: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.permissions-auto-approve", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment.", "note": "bullet 1: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.usage-cost", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment. | **Example 40 with Claude Code (`S21.live-turn`, `S21.step`, `S21.ask-agent`, `S21.usage-cost`), blocked: needs `ANTHROPIC_API_KEY` (optionally `ANTHROPIC_BASE_URL` for a LiteLLM proxy), plus `npx` and network for the image URL.** Run it from the repo root. The agent lists the `openhands-sdk/openhands/sdk/agent/` files, describes the python.org image and answers the `ask_agent` question. The example prints `EXAMPLE_COST` twice, then `Done!`.", "note": "bullet 1: pass; bullet 14: blocked (bullet is labeled blocked)", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.mcp-forwarding", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment.", "note": "bullet 1: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.secrets-env", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment. | **A Codex binary with file credentials (`S21.file-secrets`, `S21.auth-selection`, `S21.model-selection`, `S21.secrets-env`).** The server is launched by path as `/bin/codex-acp`, the form a pinned install takes. It reports `codex-acp` and offers the auth methods `chat-gpt` and `api-key`. `acp_model=\"gpt-5.5/high\"` (the legacy `/` form), and `CODEX_HOME` points at an ambient directory. There are two runs:", "note": "bullet 1: pass; bullet 8: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.masking", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment.", "note": "bullet 1: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.model-selection", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment. | **Resume a persisted session (`S21.session-resume`, `S21.model-selection`).** A conversation with `persistence_dir` runs a turn and closes. The program then reopens it by id with a new `ACPAgent` (a new subprocess). A third agent carries `acp_resume_session_id` with no saved state, and a fourth an id the server has forgotten. | **A Codex binary with file credentials (`S21.file-secrets`, `S21.auth-selection`, `S21.model-selection`, `S21.secrets-env`).** The server is launched by path as `/bin/codex-acp`, the form a pinned install takes. It reports `codex-acp` and offers the auth methods `chat-gpt` and `api-key`. `acp_model=\"gpt-5.5/high\"` (the legacy `/` form), and `CODEX_HOME` points at an ambient directory. There are two runs:", "note": "bullet 1: pass; bullet 3: pass; bullet 8: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.model-switch", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment.", "note": "bullet 1: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.init-state", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment. | **Startup failures (`S21.init-state`).** Four cases:", "note": "bullet 1: pass; bullet 4: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.cleanup", "result": "pass", "proof": ["scripted"], "entry": "**A conversation through an ACP server (`S21.construct`, `S21.step`, `S21.tool-call-events`, `S21.permissions-auto-approve`, `S21.usage-cost`, `S21.mcp-forwarding`, `S21.secrets-env`, `S21.masking`, `S21.model-selection`, `S21.model-switch`, `S21.init-state`, `S21.cleanup`).** The program builds `ACPAgent(acp_command=..., acp_model=\"m2\", mcp_config={stdio server, http server with headers}, agent_context=a skill)`. It runs a `Conversation` with `secrets={\"FAKE_TOKEN\": ..., \"EMPTY_SECRET\": \"\"}` and a token callback, sending one text + inline PNG turn and then a `[secret]` turn whose reply, thought and tool output echo the secret. It then switches the model with `switch_acp_model(\"m1\")`, tries an unknown model, closes the agent twice, and switches the model on a second conversation before its first run. `npm_config_*` and `CLAUDECODE` are set in the parent's environment.", "note": "bullet 1: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-01.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.ask-agent", "result": "fail", "proof": [], "entry": "**`ask_agent` forks the session (`S21.ask-agent`).** Same program. After the two turns, the conversation's `ask_agent` forks the main session and prompts the fork; the fork answers and reports `usage_update` (cost 0.05) plus token usage. `ACPAgent.ask_agent` on an agent with no session raises. | **Example 40 with Claude Code (`S21.live-turn`, `S21.step`, `S21.ask-agent`, `S21.usage-cost`), blocked: needs `ANTHROPIC_API_KEY` (optionally `ANTHROPIC_BASE_URL` for a LiteLLM proxy), plus `npx` and network for the image URL.** Run it from the repo root. The agent lists the `openhands-sdk/openhands/sdk/agent/` files, describes the python.org image and answers the `ask_agent` question. The example prints `EXAMPLE_COST` twice, then `Done!`.", "note": "bullet 2: fail (block 1 exited 1); bullet 14: blocked (bullet is labeled blocked)", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-02.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.session-resume", "result": "pass", "proof": ["scripted"], "entry": "**Resume a persisted session (`S21.session-resume`, `S21.model-selection`).** A conversation with `persistence_dir` runs a turn and closes. The program then reopens it by id with a new `ACPAgent` (a new subprocess). A third agent carries `acp_resume_session_id` with no saved state, and a fourth an id the server has forgotten.", "note": "bullet 3: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-03.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.timeouts-retry", "result": "fail", "proof": ["scripted"], "entry": "**Idle timeout and retries (`S21.timeouts-retry`).** `acp_prompt_timeout=2`: | **A server that goes silent (`S21.timeouts-retry`).** `[hang]` sends nothing until it is cancelled.", "note": "bullet 5: pass; bullet 6: fail (block 1 exited 1)", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-06.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.cancel-pause", "result": "pass", "proof": ["scripted"], "entry": "**Cancel a running turn (`S21.cancel-pause`).** `arun()` runs as a task. The server starts a long tool call and waits, and the program cancels the task. Then it runs another turn.", "note": "bullet 7: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-07.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.file-secrets", "result": "pass", "proof": ["scripted"], "entry": "**A Codex binary with file credentials (`S21.file-secrets`, `S21.auth-selection`, `S21.model-selection`, `S21.secrets-env`).** The server is launched by path as `/bin/codex-acp`, the form a pinned install takes. It reports `codex-acp` and offers the auth methods `chat-gpt` and `api-key`. `acp_model=\"gpt-5.5/high\"` (the legacy `/` form), and `CODEX_HOME` points at an ambient directory. There are two runs:", "note": "bullet 8: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-08.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.auth-selection", "result": "pass", "proof": ["scripted"], "entry": "**A Codex binary with file credentials (`S21.file-secrets`, `S21.auth-selection`, `S21.model-selection`, `S21.secrets-env`).** The server is launched by path as `/bin/codex-acp`, the form a pinned install takes. It reports `codex-acp` and offers the auth methods `chat-gpt` and `api-key`. `acp_model=\"gpt-5.5/high\"` (the legacy `/` form), and `CODEX_HOME` points at an ambient directory. There are two runs:", "note": "bullet 8: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-08.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.model-discovery", "result": "pass", "proof": ["scripted"], "entry": "**Model discovery and settings (`S21.model-discovery`, `S21.settings`).** `discover_acp_models(ACPAgentSettings(acp_server=\"custom\", acp_command=...))` runs against: | **Reasoning effort (`S21.effort`, `S21.effort-switch`, `S21.model-discovery`, `S21.settings`).** `programs/S21/acp_effort.py` runs `fake_acp.py` with `FAKE_ACP_EFFORT=1`: model `m1` offers a `thought_level` select (`effort`: `low`/`medium`/`high`, server default `medium`, reset by a model switch) and `m2` offers none. It drives:", "note": "bullet 9: pass; bullet 10: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-09.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.settings", "result": "pass", "proof": ["scripted"], "entry": "**Model discovery and settings (`S21.model-discovery`, `S21.settings`).** `discover_acp_models(ACPAgentSettings(acp_server=\"custom\", acp_command=...))` runs against: | **Reasoning effort (`S21.effort`, `S21.effort-switch`, `S21.model-discovery`, `S21.settings`).** `programs/S21/acp_effort.py` runs `fake_acp.py` with `FAKE_ACP_EFFORT=1`: model `m1` offers a `thought_level` select (`effort`: `low`/`medium`/`high`, server default `medium`, reset by a model switch) and `m2` offers none. It drives:", "note": "bullet 9: pass; bullet 10: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-09.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.effort", "result": "pass", "proof": ["scripted"], "entry": "**Reasoning effort (`S21.effort`, `S21.effort-switch`, `S21.model-discovery`, `S21.settings`).** `programs/S21/acp_effort.py` runs `fake_acp.py` with `FAKE_ACP_EFFORT=1`: model `m1` offers a `thought_level` select (`effort`: `low`/`medium`/`high`, server default `medium`, reset by a model switch) and `m2` offers none. It drives:", "note": "bullet 10: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-10.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.effort-switch", "result": "pass", "proof": ["scripted"], "entry": "**Reasoning effort (`S21.effort`, `S21.effort-switch`, `S21.model-discovery`, `S21.settings`).** `programs/S21/acp_effort.py` runs `fake_acp.py` with `FAKE_ACP_EFFORT=1`: model `m1` offers a `thought_level` select (`effort`: `low`/`medium`/`high`, server default `medium`, reset by a model switch) and `m2` offers none. It drives:", "note": "bullet 10: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-10.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.tracing", "result": "pass", "proof": ["scripted"], "entry": "**Traces of an ACP turn (`S21.tracing`).** `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` points at the local collector from `programs/S20/otlp.py`. The user's message contains the secret value.", "note": "bullet 11: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-11.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.provider-registry", "result": "pass", "proof": ["scripted"], "entry": "**Provider registry (`S21.provider-registry`).** Reuses the S19 catalog program: launch commands pin the catalog versions, and detection works by agent name and by command. Then, over the network, every pinned package version must exist on npm. | **Live handshake with every built-in provider (`S21.handshake-live`, `S21.provider-registry`).** Network, no account. For each provider, `ACPAgentSettings(acp_server=KEY)` resolves the default `npx` command, and `discover_acp_models` runs the real server's handshake with a bogus API key. kimi-code also gets `KIMI_MODEL_NAME`/`KIMI_MODEL_API_KEY`, and opencode gets `OPENCODE_API_KEY`.", "note": "bullet 12: pass; bullet 13: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-12.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.handshake-live", "result": "pass", "proof": ["scripted"], "entry": "**Live handshake with every built-in provider (`S21.handshake-live`, `S21.provider-registry`).** Network, no account. For each provider, `ACPAgentSettings(acp_server=KEY)` resolves the default `npx` command, and `discover_acp_models` runs the real server's handshake with a bogus API key. kimi-code also gets `KIMI_MODEL_NAME`/`KIMI_MODEL_API_KEY`, and opencode gets `OPENCODE_API_KEY`.", "note": "bullet 13: pass", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": "/sdk/S21-effort/transcripts/S21-acp-agent-13.log", "at": "2026-11-11T08:39:16+00:00"} +{"feature": "S21.live-turn", "result": "blocked", "proof": [], "entry": "**Example 40 with Claude Code (`S21.live-turn`, `S21.step`, `S21.ask-agent`, `S21.usage-cost`), blocked: needs `ANTHROPIC_API_KEY` (optionally `ANTHROPIC_BASE_URL` for a LiteLLM proxy), plus `npx` and network for the image URL.** Run it from the repo root. The agent lists the `openhands-sdk/openhands/sdk/agent/` files, describes the python.org image and answers the `ask_agent` question. The example prints `EXAMPLE_COST` twice, then `Done!`.", "note": "bullet 14: blocked (bullet is labeled blocked)", "revision": "07e3bb94b79205c1bb5c12dd1965ccaecd2f8e72", "transcript": null, "at": "2026-11-11T08:39:16+00:00"} diff --git a/.pr/map-maintenance/2026-11-11/sdk-report.md b/.pr/map-maintenance/2026-11-11/sdk-report.md new file mode 100644 index 00000000000..ad0330371df --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/sdk-report.md @@ -0,0 +1,491 @@ +## Conflicting attempts + +- S04.persist-events: fail at 2026-11-11T09:08:40+00:00, pass at 2026-11-11T09:22:17+00:00, pass at 2026-11-11T09:29:38+00:00 +- S05.event-log-recovery: pass at 2026-11-11T09:06:44+00:00, fail at 2026-11-11T09:08:40+00:00, pass at 2026-11-11T09:29:38+00:00 +- S06.condenser-resume: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.context-overflow-request: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.context-skills: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.context-suffixes: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.hard-reset: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.memory-load: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.memory-write-path: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.soft-vs-hard: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.summarizing-size: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.summarizing-tokens: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S06.summary-llm-streaming: fail at 2026-11-11T09:13:15+00:00, pass at 2026-11-11T09:16:43+00:00 +- S11.builtin-agents: fail at 2026-11-11T09:14:09+00:00, pass at 2026-11-11T09:18:21+00:00 +- S11.definition-factory: fail at 2026-11-11T09:14:09+00:00, pass at 2026-11-11T09:18:21+00:00 +- S11.delegation-visualizer: fail at 2026-11-11T09:14:09+00:00, pass at 2026-11-11T09:18:21+00:00 +- S11.register-programmatic: fail at 2026-11-11T09:14:09+00:00, pass at 2026-11-11T09:18:21+00:00 +- S11.task-metrics: fail at 2026-11-11T09:14:09+00:00, pass at 2026-11-11T09:18:21+00:00 +- S11.task-resume: fail at 2026-11-11T09:14:09+00:00, pass at 2026-11-11T09:18:21+00:00 +- S11.task-run: fail at 2026-11-11T09:14:09+00:00, pass at 2026-11-11T09:18:21+00:00 +- S21.auth-selection: fail at 2026-11-11T08:27:22+00:00, pass at 2026-11-11T08:39:16+00:00 +- S21.file-secrets: fail at 2026-11-11T08:27:22+00:00, pass at 2026-11-11T08:39:16+00:00 +- S21.model-selection: fail at 2026-11-11T08:27:22+00:00, pass at 2026-11-11T08:39:16+00:00 +- S21.secrets-env: fail at 2026-11-11T08:27:22+00:00, pass at 2026-11-11T08:39:16+00:00 + +## Checks + +| Feature | Result | Revision | Note | Transcript | +|---|---|---|---|---| +| S01.stuck-recover | fail | 5ff99f1b6ca2 | bullet 7: pass; bullet 8: fail (block 1 exited 1) | /sdk/S01/transcripts/S01-agent-loop-08.log | +| S02.no-response-retry | fail | 391b5546481d | bullet 9: fail (block 1 exited 1) | /sdk/S02/transcripts/S02-llm-core-09.log | +| S03.registry-stats-wiring | fail | 391b5546481d | bullet 1: pass; bullet 2: fail (block 1 exited 1); bullet 15: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-02.log | +| S04.interactive-terminal | fail | 391b5546481d | bullet 19: fail (block 1 exited 1); bullet 25: pass | /sdk/S04/transcripts/S04-local-conversation-19.log | +| S07.available-skills-prompt | fail | e776f878876e | bullet 2: pass; bullet 3: fail (block 1 exited 1); bullet 10: pass; bullet 15: pass | /sdk/S07/transcripts/S07-skills-03.log | +| S07.installed-lifecycle | fail | e776f878876e | bullet 6: pass; bullet 13: fail (block 1 exited 1) | /sdk/S07/transcripts/S07-skills-13.log | +| S08.secret-expansion | fail | edff6711505a | bullet 3: fail (block 1 exited 1) | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-03.log | +| S09.session-recovery | fail | edff6711505a | bullet 10: pass; bullet 11: fail (block 1 exited 1) | /sdk/S09/transcripts/S09-mcp-11.log | +| S11.factory-info | fail | c5f588f089d0 | bullet 5: pass; bullet 6: fail (block 1 exited 1) | /sdk/S11-nokey/transcripts/S11-subagents-delegation-06.log | +| S11.task-observability | fail | c5f588f089d0 | bullet 2: fail (block 1 exited 1); bullet 3: blocked (bullet is labeled blocked) | /sdk/S11-nokey/transcripts/S11-subagents-delegation-02.log | +| S14.terminal-backend-select | fail | c707d3737e8d | bullet 2: pass; bullet 3: pass; bullet 6: fail (block 1 exited 1) | /sdk/S14-nokey/transcripts/S14-core-tools-06.log | +| S14.terminal-input-encoding | fail | c707d3737e8d | bullet 2: pass; bullet 5: fail (block 1 exited 1) | /sdk/S14-nokey/transcripts/S14-core-tools-05.log | +| S14.terminal-reset | fail | c707d3737e8d | bullet 2: pass; bullet 3: pass; bullet 6: fail (block 1 exited 1) | /sdk/S14-nokey/transcripts/S14-core-tools-06.log | +| S15.action-routing | fail | c707d3737e8d | bullet 1: pass; bullet 5: fail (block 1 exited 1) | /sdk/S15-nokey/transcripts/S15-browser-05.log | +| S15.cleanup | fail | c707d3737e8d | bullet 1: pass; bullet 3: pass; bullet 5: fail (block 1 exited 1) | /sdk/S15-nokey/transcripts/S15-browser-05.log | +| S15.session-recovery | fail | c707d3737e8d | bullet 5: fail (block 1 exited 1) | /sdk/S15-nokey/transcripts/S15-browser-05.log | +| S18.workspace-factory | fail | 1ecd2de9dc6b | bullet 1: pass; bullet 3: fail (block 1 exited 1) | /sdk/S18-nokey/transcripts/S18-workspace-backends-03.log | +| S21.ask-agent | fail | 07e3bb94b792 | bullet 2: fail (block 1 exited 1); bullet 14: blocked (bullet is labeled blocked) | /sdk/S21-effort/transcripts/S21-acp-agent-02.log | +| S21.timeouts-retry | fail | 07e3bb94b792 | bullet 5: pass; bullet 6: fail (block 1 exited 1) | /sdk/S21-effort/transcripts/S21-acp-agent-06.log | +| S05.token-event | blocked | e776f878876e | bullet 9: blocked (bullet is labeled blocked) | | +| S14.terminal-windows | blocked | c707d3737e8d | bullet 7: blocked (bullet is labeled blocked) | | +| S15.chromium-detection | blocked | c707d3737e8d | bullet 2: blocked (block 1 is blocked (exit 3: a missing prerequisite)) | /sdk/S15-nokey/transcripts/S15-browser-02.log | +| S15.degrade-on-failure | blocked | c707d3737e8d | bullet 2: blocked (block 1 is blocked (exit 3: a missing prerequisite)) | /sdk/S15-nokey/transcripts/S15-browser-02.log | +| S15.recording-storage | blocked | c707d3737e8d | bullet 6: blocked (bullet is labeled blocked); bullet 9: blocked (bullet is labeled blocked) | | +| S18.agent-sandbox | blocked | 1ecd2de9dc6b | bullet 11: blocked (bullet is labeled blocked) | | +| S18.api-remote-lifecycle | blocked | 1ecd2de9dc6b | bullet 10: blocked (bullet is labeled blocked) | | +| S18.api-remote-start | blocked | 1ecd2de9dc6b | bullet 10: blocked (bullet is labeled blocked) | | +| S18.apptainer | blocked | 1ecd2de9dc6b | bullet 11: blocked (bullet is labeled blocked) | | +| S18.cloud-sandbox | blocked | 1ecd2de9dc6b | bullet 10: blocked (bullet is labeled blocked) | | +| S18.docker-dev-build | blocked | 1ecd2de9dc6b | bullet 9: blocked (bullet is labeled blocked) | | +| S18.docker-lifecycle | blocked | 1ecd2de9dc6b | bullet 9: blocked (bullet is labeled blocked) | | +| S18.docker-start | blocked | 1ecd2de9dc6b | bullet 9: blocked (bullet is labeled blocked) | | +| S21.live-turn | blocked | 07e3bb94b792 | bullet 14: blocked (bullet is labeled blocked) | | +| S01.ask-agent | pass | 5ff99f1b6ca2 | bullet 17: pass; bullet 20: pass | /sdk/S01/transcripts/S01-agent-loop-17.log | +| S01.async-step | pass | 5ff99f1b6ca2 | bullet 15: pass | /sdk/S01/transcripts/S01-agent-loop-15.log | +| S01.content-policy | pass | 5ff99f1b6ca2 | bullet 3: pass | /sdk/S01/transcripts/S01-agent-loop-03.log | +| S01.context-overflow | pass | 5ff99f1b6ca2 | bullet 19: pass; bullet 23: pass | /sdk/S01/transcripts/S01-agent-loop-19.log | +| S01.empty-reply-nudge | pass | 5ff99f1b6ca2 | bullet 3: pass; bullet 4: pass | /sdk/S01/transcripts/S01-agent-loop-03.log | +| S01.finish-truncates-batch | pass | 5ff99f1b6ca2 | bullet 11: pass | /sdk/S01/transcripts/S01-agent-loop-11.log | +| S01.image-input | pass | 5ff99f1b6ca2 | bullet 16: pass; bullet 24: blocked (bullet is labeled blocked) | /sdk/S01/transcripts/S01-agent-loop-16.log | +| S01.image-non-vision | pass | 5ff99f1b6ca2 | bullet 16: pass | /sdk/S01/transcripts/S01-agent-loop-16.log | +| S01.iterative-refinement | pass | 5ff99f1b6ca2 | bullet 22: pass | /sdk/S01/transcripts/S01-agent-loop-22.log | +| S01.malformed-tool-call | pass | 5ff99f1b6ca2 | bullet 3: pass | /sdk/S01/transcripts/S01-agent-loop-03.log | +| S01.max-budget | pass | 5ff99f1b6ca2 | bullet 6: pass | /sdk/S01/transcripts/S01-agent-loop-06.log | +| S01.max-iterations | pass | 5ff99f1b6ca2 | bullet 5: pass | /sdk/S01/transcripts/S01-agent-loop-05.log | +| S01.message-while-running | pass | 5ff99f1b6ca2 | bullet 14: pass; bullet 15: pass; bullet 21: pass | /sdk/S01/transcripts/S01-agent-loop-14.log | +| S01.parallel-resource-locks | pass | 5ff99f1b6ca2 | bullet 12: pass | /sdk/S01/transcripts/S01-agent-loop-12.log | +| S01.parallel-tools | pass | 5ff99f1b6ca2 | bullet 11: pass; bullet 21: pass | /sdk/S01/transcripts/S01-agent-loop-11.log | +| S01.response-dispatch | pass | 5ff99f1b6ca2 | bullet 1: pass; bullet 2: pass; bullet 20: pass | /sdk/S01/transcripts/S01-agent-loop-01.log | +| S01.run-to-finish | pass | 5ff99f1b6ca2 | bullet 1: pass; bullet 20: pass; bullet 23: pass | /sdk/S01/transcripts/S01-agent-loop-01.log | +| S01.structured-output | pass | 391b5546481d | bullet 18: pass; bullet 21: pass | /sdk/s01fix/r1/transcripts/S01-agent-loop-18.log | +| S01.stuck-detect | pass | 5ff99f1b6ca2 | bullet 7: pass; bullet 21: pass | /sdk/S01/transcripts/S01-agent-loop-07.log | +| S01.stuck-nudge | pass | 5ff99f1b6ca2 | bullet 10: pass | /sdk/S01/transcripts/S01-agent-loop-10.log | +| S01.stuck-thresholds | pass | 5ff99f1b6ca2 | bullet 9: pass | /sdk/S01/transcripts/S01-agent-loop-09.log | +| S01.tool-exception | pass | 5ff99f1b6ca2 | bullet 13: pass | /sdk/S01/transcripts/S01-agent-loop-13.log | +| S01.unknown-tool | pass | 5ff99f1b6ca2 | bullet 3: pass | /sdk/S01/transcripts/S01-agent-loop-03.log | +| S02.anthropic-thinking | pass | 391b5546481d | bullet 5: pass; bullet 20: blocked (bullet is labeled blocked) | /sdk/S02/transcripts/S02-llm-core-05.log | +| S02.api-mode | pass | 391b5546481d | bullet 1: pass; bullet 2: pass; bullet 17: pass | /sdk/S02/transcripts/S02-llm-core-01.log | +| S02.completion | pass | 391b5546481d | bullet 1: pass; bullet 17: pass | /sdk/S02/transcripts/S02-llm-core-01.log | +| S02.context-window-guard | pass | 391b5546481d | bullet 12: pass | /sdk/S02/transcripts/S02-llm-core-12.log | +| S02.conversation-stats | pass | 391b5546481d | bullet 16: pass; bullet 18: pass | /sdk/S02/transcripts/S02-llm-core-16.log | +| S02.error-mapping | pass | e776f878876e | bullet 7: pass | /sdk/S05/transcripts/S05-events-07.log | +| S02.image-inline | pass | 391b5546481d | bullet 14: pass | /sdk/S02/transcripts/S02-llm-core-14.log | +| S02.log-completions | pass | 391b5546481d | bullet 1: pass | /sdk/S02/transcripts/S02-llm-core-01.log | +| S02.message-formatting | pass | 391b5546481d | bullet 13: pass; bullet 18: pass | /sdk/S02/transcripts/S02-llm-core-13.log | +| S02.metrics | pass | 391b5546481d | bullet 1: pass; bullet 16: pass; bullet 17: pass; bullet 18: pass | /sdk/S02/transcripts/S02-llm-core-01.log | +| S02.model-features | pass | 391b5546481d | bullet 6: pass | /sdk/S02/transcripts/S02-llm-core-06.log | +| S02.native-tool-calling | pass | 391b5546481d | bullet 1: pass; bullet 7: pass | /sdk/S02/transcripts/S02-llm-core-01.log | +| S02.pricing-override | pass | 391b5546481d | bullet 1: pass | /sdk/S02/transcripts/S02-llm-core-01.log | +| S02.prompt-caching | pass | 391b5546481d | bullet 5: pass; bullet 11: pass | /sdk/S02/transcripts/S02-llm-core-05.log | +| S02.prompted-tool-calling | pass | 391b5546481d | bullet 7: pass | /sdk/S02/transcripts/S02-llm-core-07.log | +| S02.reasoning-content | pass | 391b5546481d | bullet 1: pass; bullet 3: pass | /sdk/S02/transcripts/S02-llm-core-01.log | +| S02.reasoning-options | pass | 391b5546481d | bullet 6: pass | /sdk/S02/transcripts/S02-llm-core-06.log | +| S02.responses-reasoning | pass | 391b5546481d | bullet 2: pass; bullet 20: blocked (bullet is labeled blocked) | /sdk/S02/transcripts/S02-llm-core-02.log | +| S02.retries | pass | 391b5546481d | bullet 8: pass | /sdk/S02/transcripts/S02-llm-core-08.log | +| S02.serialization | pass | 391b5546481d | bullet 15: pass | /sdk/S02/transcripts/S02-llm-core-15.log | +| S02.stream-no-callback | pass | 391b5546481d | bullet 3: pass | /sdk/S02/transcripts/S02-llm-core-03.log | +| S02.streaming | pass | 391b5546481d | bullet 3: pass; bullet 17: pass; bullet 18: pass | /sdk/S02/transcripts/S02-llm-core-03.log | +| S02.timeouts | pass | 391b5546481d | bullet 4: pass | /sdk/S02/transcripts/S02-llm-core-04.log | +| S02.token-count | pass | 391b5546481d | bullet 12: pass | /sdk/S02/transcripts/S02-llm-core-12.log | +| S03.agent-llm-discovery | pass | 391b5546481d | bullet 3: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-03.log | +| S03.fallback-metrics | pass | 391b5546481d | bullet 6: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-06.log | +| S03.fallback-profile-resolution | pass | 391b5546481d | bullet 6: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-06.log | +| S03.fallback-skip | pass | 391b5546481d | bullet 6: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-06.log | +| S03.fallback-transient | pass | 391b5546481d | bullet 6: pass; bullet 16: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-06.log | +| S03.meta-profile-store | pass | 391b5546481d | bullet 13: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-13.log | +| S03.multimodal-router-images | pass | 391b5546481d | bullet 4: pass; bullet 17: blocked (bullet is labeled blocked) | /sdk/S03/transcripts/S03-llm-routing-profiles-04.log | +| S03.multimodal-router-text | pass | 391b5546481d | bullet 4: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-04.log | +| S03.profile-concurrency-limits | pass | 391b5546481d | bullet 7: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-07.log | +| S03.profile-migration | pass | 391b5546481d | bullet 7: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-07.log | +| S03.profile-save-load | pass | 391b5546481d | bullet 7: pass; bullet 16: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-07.log | +| S03.profile-secrets | pass | 391b5546481d | bullet 7: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-07.log | +| S03.provider-connections | pass | 391b5546481d | bullet 8: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-08.log | +| S03.random-router | pass | 391b5546481d | bullet 5: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-05.log | +| S03.registry-crud | pass | 391b5546481d | bullet 1: pass; bullet 16: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-01.log | +| S03.registry-independent-metrics | pass | 391b5546481d | bullet 1: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-01.log | +| S03.route-task-to-model | pass | 391b5546481d | bullet 12: pass; bullet 17: blocked (bullet is labeled blocked) | /sdk/S03/transcripts/S03-llm-routing-profiles-12.log | +| S03.router-validation | pass | 391b5546481d | bullet 5: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-05.log | +| S03.subscription-login | pass | 391b5546481d | bullet 14: pass; bullet 18: blocked (bullet is labeled blocked) | /sdk/S03/transcripts/S03-llm-routing-profiles-14.log | +| S03.subscription-requests | pass | 391b5546481d | bullet 14: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-14.log | +| S03.switch-llm-object | pass | 391b5546481d | bullet 10: pass | /sdk/S03/transcripts/S03-llm-routing-profiles-10.log | +| S03.switch-llm-tool | pass | 391b5546481d | bullet 11: pass; bullet 17: blocked (bullet is labeled blocked) | /sdk/S03/transcripts/S03-llm-routing-profiles-11.log | +| S03.switch-profile | pass | 391b5546481d | bullet 9: pass; bullet 15: pass; bullet 17: blocked (bullet is labeled blocked) | /sdk/S03/transcripts/S03-llm-routing-profiles-09.log | +| S04.arun-interrupt | pass | 391b5546481d | bullet 12: pass; bullet 24: pass | /sdk/S04/transcripts/S04-local-conversation-12.log | +| S04.async-callback | pass | 391b5546481d | bullet 11: pass; bullet 23: pass | /sdk/S04/transcripts/S04-local-conversation-11.log | +| S04.callbacks | pass | 391b5546481d | bullet 2: pass; bullet 7: pass | /sdk/S04/transcripts/S04-local-conversation-02.log | +| S04.cancel-token | pass | 391b5546481d | bullet 13: pass; bullet 24: pass | /sdk/S04/transcripts/S04-local-conversation-13.log | +| S04.close | pass | 391b5546481d | bullet 9: pass | /sdk/S04/transcripts/S04-local-conversation-09.log | +| S04.conversation-id | pass | 391b5546481d | bullet 1: pass | /sdk/S04/transcripts/S04-local-conversation-01.log | +| S04.custom-file-store | pass | 391b5546481d | bullet 6: pass | /sdk/S04/transcripts/S04-local-conversation-06.log | +| S04.factory | pass | 391b5546481d | bullet 1: pass; bullet 22: pass; bullet 26: blocked (bullet is labeled blocked) | /sdk/S04/transcripts/S04-local-conversation-01.log | +| S04.final-response | pass | 391b5546481d | bullet 20: pass; bullet 22: pass | /sdk/S04/transcripts/S04-local-conversation-20.log | +| S04.fork | pass | 391b5546481d | bullet 16: pass; bullet 24: pass | /sdk/S04/transcripts/S04-local-conversation-16.log | +| S04.fork-branch | pass | 391b5546481d | bullet 17: pass; bullet 24: pass | /sdk/S04/transcripts/S04-local-conversation-17.log | +| S04.goal-loop | pass | 391b5546481d | bullet 18: pass; bullet 24: pass | /sdk/S04/transcripts/S04-local-conversation-18.log | +| S04.pause | pass | 391b5546481d | bullet 10: pass; bullet 23: pass | /sdk/S04/transcripts/S04-local-conversation-10.log | +| S04.persist-events | pass | 0da9cf850a0e | bullet 2: pass; bullet 3: pass; bullet 23: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S04-persist2/transcripts/S04-local-conversation-02.log | +| S04.resume | pass | 391b5546481d | bullet 2: pass; bullet 4: pass; bullet 22: pass; bullet 23: pass | /sdk/S04/transcripts/S04-local-conversation-02.log | +| S04.resume-agent-compat | pass | 391b5546481d | bullet 5: pass | /sdk/S04/transcripts/S04-local-conversation-05.log | +| S04.run-error | pass | 391b5546481d | bullet 8: pass | /sdk/S04/transcripts/S04-local-conversation-08.log | +| S04.secrets-constructor | pass | 391b5546481d | bullet 14: pass | /sdk/S04/transcripts/S04-local-conversation-14.log | +| S04.secrets-masking | pass | 391b5546481d | bullet 14: pass; bullet 23: pass | /sdk/S04/transcripts/S04-local-conversation-14.log | +| S04.secrets-update | pass | 391b5546481d | bullet 14: pass; bullet 23: pass | /sdk/S04/transcripts/S04-local-conversation-14.log | +| S04.stats | pass | 391b5546481d | bullet 21: pass | /sdk/S04/transcripts/S04-local-conversation-21.log | +| S04.title-and-tags | pass | 391b5546481d | bullet 20: pass; bullet 24: pass | /sdk/S04/transcripts/S04-local-conversation-20.log | +| S04.visualizer-custom | pass | 391b5546481d | bullet 15: pass; bullet 24: pass | /sdk/S04/transcripts/S04-local-conversation-15.log | +| S04.visualizer-default | pass | 391b5546481d | bullet 15: pass; bullet 23: pass | /sdk/S04/transcripts/S04-local-conversation-15.log | +| S05.completion-log-event | pass | e776f878876e | bullet 2: pass | /sdk/S05/transcripts/S05-events-02.log | +| S05.error-classification | pass | e776f878876e | bullet 2: pass | /sdk/S05/transcripts/S05-events-02.log | +| S05.event-base | pass | e776f878876e | bullet 1: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.event-cache | pass | e776f878876e | bullet 1: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.event-log-append | pass | e776f878876e | bullet 3: pass | /sdk/S05/transcripts/S05-events-03.log | +| S05.event-log-files | pass | e776f878876e | bullet 1: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.event-log-recovery | pass | 0da9cf850a0e | bullet 3: pass | /sdk/S04-persist2/transcripts/S04-local-conversation-03.log | +| S05.event-tree | pass | e776f878876e | bullet 1: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.events-to-messages | pass | e776f878876e | bullet 1: pass; bullet 5: pass; bullet 7: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.json-roundtrip | pass | e776f878876e | bullet 1: pass; bullet 5: pass; bullet 6: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.openai-format | pass | e776f878876e | bullet 1: pass; bullet 5: pass; bullet 6: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.resume-transcript | pass | e776f878876e | bullet 1: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.streaming-delta | pass | e776f878876e | bullet 2: pass | /sdk/S05/transcripts/S05-events-02.log | +| S05.system-prompt-blocks | pass | e776f878876e | bullet 8: pass | /sdk/S05/transcripts/S05-events-08.log | +| S05.to-llm-message | pass | e776f878876e | bullet 1: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.unmatched-actions | pass | e776f878876e | bullet 1: pass | /sdk/S05/transcripts/S05-events-01.log | +| S05.visualize | pass | e776f878876e | bullet 1: pass | /sdk/S05/transcripts/S05-events-01.log | +| S06.condenser-resume | pass | edff6711505a | bullet 11: pass; bullet 14: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-11.log | +| S06.context-datetime | pass | edff6711505a | bullet 1: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-01.log | +| S06.context-overflow-request | pass | edff6711505a | bullet 10: pass; bullet 17: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-10.log | +| S06.context-secrets | pass | edff6711505a | bullet 1: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-01.log | +| S06.context-skills | pass | edff6711505a | bullet 1: pass; bullet 16: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-01.log | +| S06.context-suffixes | pass | edff6711505a | bullet 1: pass; bullet 16: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-01.log | +| S06.hard-reset | pass | edff6711505a | bullet 8: pass; bullet 17: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-08.log | +| S06.manual-condense | pass | edff6711505a | bullet 9: pass; bullet 13: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-09.log | +| S06.memory-load | pass | edff6711505a | bullet 3: pass; bullet 15: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-03.log | +| S06.memory-write-path | pass | edff6711505a | bullet 3: pass; bullet 15: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-03.log | +| S06.noop-condenser | pass | edff6711505a | bullet 6: pass; bullet 12: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-06.log | +| S06.persona-soul | pass | edff6711505a | bullet 2: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-02.log | +| S06.pipeline-condenser | pass | edff6711505a | bullet 12: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-12.log | +| S06.soft-vs-hard | pass | edff6711505a | bullet 7: pass; bullet 17: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-07.log | +| S06.summarizing-reasons | pass | edff6711505a | bullet 6: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-06.log | +| S06.summarizing-size | pass | edff6711505a | bullet 4: pass; bullet 13: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 14: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 17: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-04.log | +| S06.summarizing-tokens | pass | edff6711505a | bullet 5: pass; bullet 17: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-05.log | +| S06.summary-llm-streaming | pass | edff6711505a | bullet 4: pass; bullet 14: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S06-nokey/transcripts/S06-context-condensers-04.log | +| S06.system-prompt-custom | pass | edff6711505a | bullet 2: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-02.log | +| S06.system-prompt-model-specific | pass | edff6711505a | bullet 2: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-02.log | +| S06.system-prompt-registry | pass | edff6711505a | bullet 2: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-02.log | +| S06.view | pass | edff6711505a | bullet 4: pass; bullet 6: pass | /sdk/S06-nokey/transcripts/S06-context-condensers-04.log | +| S06.view-atomicity | pass | edff6711505a | bullet 4: pass; bullet 13: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 18: blocked (bullet is labeled blocked) | /sdk/S06-nokey/transcripts/S06-context-condensers-04.log | +| S07.agentskills-load | pass | e776f878876e | bullet 2: pass; bullet 12: pass | /sdk/S07/transcripts/S07-skills-02.log | +| S07.agentskills-trigger | pass | e776f878876e | bullet 2: pass; bullet 12: pass | /sdk/S07/transcripts/S07-skills-02.log | +| S07.disable-model-invocation | pass | e776f878876e | bullet 2: pass | /sdk/S07/transcripts/S07-skills-02.log | +| S07.disabled-skills | pass | e776f878876e | bullet 5: pass | /sdk/S07/transcripts/S07-skills-05.log | +| S07.inline-commands | pass | e776f878876e | bullet 2: pass | /sdk/S07/transcripts/S07-skills-02.log | +| S07.invoke-skill-autoattach | pass | e776f878876e | bullet 2: pass; bullet 10: pass; bullet 12: pass | /sdk/S07/transcripts/S07-skills-02.log | +| S07.invoke-skill-run | pass | e776f878876e | bullet 2: pass; bullet 10: pass; bullet 15: pass | /sdk/S07/transcripts/S07-skills-02.log | +| S07.keyword-trigger | pass | e776f878876e | bullet 1: pass; bullet 11: pass | /sdk/S07/transcripts/S07-skills-01.log | +| S07.load-project-flag | pass | e776f878876e | bullet 5: pass | /sdk/S07/transcripts/S07-skills-05.log | +| S07.marketplace-install | pass | e776f878876e | bullet 6: pass; bullet 7: pass; bullet 14: pass | /sdk/S07/transcripts/S07-skills-06.log | +| S07.merge-precedence | pass | e776f878876e | bullet 5: pass | /sdk/S07/transcripts/S07-skills-05.log | +| S07.message-suffixes | pass | e776f878876e | bullet 1: pass; bullet 11: pass | /sdk/S07/transcripts/S07-skills-01.log | +| S07.path-rules | pass | e776f878876e | bullet 4: pass | /sdk/S07/transcripts/S07-skills-04.log | +| S07.project-skills | pass | e776f878876e | bullet 5: pass | /sdk/S07/transcripts/S07-skills-05.log | +| S07.public-skills | pass | e776f878876e | bullet 8: pass; bullet 9: pass; bullet 11: pass | /sdk/S07/transcripts/S07-skills-08.log | +| S07.repo-skill-always-on | pass | e776f878876e | bullet 1: pass; bullet 11: pass | /sdk/S07/transcripts/S07-skills-01.log | +| S07.resources | pass | e776f878876e | bullet 2: pass; bullet 12: pass | /sdk/S07/transcripts/S07-skills-02.log | +| S07.serialization | pass | e776f878876e | bullet 1: pass; bullet 2: pass | /sdk/S07/transcripts/S07-skills-01.log | +| S07.skill-mcp-json | pass | e776f878876e | bullet 2: pass | /sdk/S07/transcripts/S07-skills-02.log | +| S07.task-trigger | pass | e776f878876e | bullet 1: pass | /sdk/S07/transcripts/S07-skills-01.log | +| S07.trigger-once | pass | e776f878876e | bullet 1: pass | /sdk/S07/transcripts/S07-skills-01.log | +| S07.user-skills | pass | e776f878876e | bullet 5: pass | /sdk/S07/transcripts/S07-skills-05.log | +| S08.agent-plugins-layout | pass | edff6711505a | bullet 1: pass; bullet 16: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-01.log | +| S08.agent-plugins-mcp | pass | edff6711505a | bullet 1: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-01.log | +| S08.ambient-plugins | pass | edff6711505a | bullet 2: pass; bullet 7: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-02.log | +| S08.claude-code-layout | pass | edff6711505a | bullet 1: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-01.log | +| S08.commands-as-skills | pass | edff6711505a | bullet 1: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-01.log | +| S08.conversation-attach | pass | edff6711505a | bullet 2: pass; bullet 4: pass; bullet 15: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 16: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-02.log | +| S08.fetch-cache | pass | edff6711505a | bullet 4: pass; bullet 5: pass; bullet 6: pass; bullet 15: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-04.log | +| S08.installation-manager | pass | edff6711505a | bullet 9: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-09.log | +| S08.installed-plugins | pass | edff6711505a | bullet 7: pass; bullet 8: pass; bullet 15: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-07.log | +| S08.load-plugin-runtime | pass | edff6711505a | bullet 13: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-13.log | +| S08.marketplace-auto-load | pass | edff6711505a | bullet 12: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-12.log | +| S08.marketplace-import-path | pass | edff6711505a | bullet 14: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-14.log | +| S08.marketplace-manifest | pass | edff6711505a | bullet 10: pass; bullet 17: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-10.log | +| S08.marketplace-registry | pass | edff6711505a | bullet 11: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-11.log | +| S08.merge-semantics | pass | edff6711505a | bullet 1: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-01.log | +| S08.path-containment | pass | edff6711505a | bullet 1: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-01.log | +| S08.plugin-agents | pass | edff6711505a | bullet 2: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-02.log | +| S08.resolved-refs | pass | edff6711505a | bullet 2: pass; bullet 15: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-02.log | +| S08.source-parsing | pass | edff6711505a | bullet 1: pass | /sdk/S08-nokey/transcripts/S08-plugins-marketplace-01.log | +| S09.argument-validation | pass | edff6711505a | bullet 1: pass; bullet 12: pass | /sdk/S09/transcripts/S09-mcp-01.log | +| S09.config-model | pass | edff6711505a | bullet 12: pass | /sdk/S09/transcripts/S09-mcp-12.log | +| S09.config-secrets | pass | edff6711505a | bullet 1: pass; bullet 12: pass | /sdk/S09/transcripts/S09-mcp-01.log | +| S09.config-var-expansion | pass | edff6711505a | bullet 1: pass | /sdk/S09/transcripts/S09-mcp-01.log | +| S09.enabled-flag | pass | edff6711505a | bullet 4: pass | /sdk/S09/transcripts/S09-mcp-04.log | +| S09.filter-tools-regex | pass | edff6711505a | bullet 2: pass; bullet 14: pass | /sdk/S09/transcripts/S09-mcp-02.log | +| S09.http-sse-server | pass | edff6711505a | bullet 6: pass | /sdk/S09/transcripts/S09-mcp-06.log | +| S09.listing-timeout | pass | edff6711505a | bullet 5: pass | /sdk/S09/transcripts/S09-mcp-05.log | +| S09.oauth | pass | edff6711505a | bullet 8: pass; bullet 9: blocked (bullet is labeled blocked) | /sdk/S09/transcripts/S09-mcp-08.log | +| S09.observation | pass | edff6711505a | bullet 1: pass; bullet 12: pass; bullet 13: pass; bullet 14: pass | /sdk/S09/transcripts/S09-mcp-01.log | +| S09.schema-conversion | pass | edff6711505a | bullet 1: pass; bullet 12: pass; bullet 13: pass | /sdk/S09/transcripts/S09-mcp-01.log | +| S09.secret-refs-in-calls | pass | edff6711505a | bullet 1: pass; bullet 13: pass | /sdk/S09/transcripts/S09-mcp-01.log | +| S09.session-reuse | pass | edff6711505a | bullet 1: pass | /sdk/S09/transcripts/S09-mcp-01.log | +| S09.startup-failure-tolerated | pass | edff6711505a | bullet 3: pass | /sdk/S09/transcripts/S09-mcp-03.log | +| S09.static-auth | pass | edff6711505a | bullet 7: pass | /sdk/S09/transcripts/S09-mcp-07.log | +| S09.stdio-server | pass | edff6711505a | bullet 1: pass; bullet 6: pass; bullet 13: pass; bullet 14: pass | /sdk/S09/transcripts/S09-mcp-01.log | +| S09.tool-serialization | pass | edff6711505a | bullet 12: pass | /sdk/S09/transcripts/S09-mcp-12.log | +| S09.tools-list-changed | pass | edff6711505a | bullet 1: pass | /sdk/S09/transcripts/S09-mcp-01.log | +| S10.agent-hook | pass | c5f588f089d0 | bullet 4: pass; bullet 7: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S10-nokey/transcripts/S10-hooks-04.log | +| S10.async-hooks | pass | c5f588f089d0 | bullet 1: pass | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.blocked-state-persistence | pass | c5f588f089d0 | bullet 2: pass | /sdk/S10-nokey/transcripts/S10-hooks-02.log | +| S10.command-exec | pass | c5f588f089d0 | bullet 1: pass; bullet 3: pass; bullet 6: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.config-discovery | pass | c5f588f089d0 | bullet 1: pass; bullet 3: pass | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.config-format | pass | c5f588f089d0 | bullet 3: pass | /sdk/S10-nokey/transcripts/S10-hooks-03.log | +| S10.definition-validation | pass | c5f588f089d0 | bullet 3: pass | /sdk/S10-nokey/transcripts/S10-hooks-03.log | +| S10.execution-events | pass | c5f588f089d0 | bullet 1: pass; bullet 7: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.exit-codes | pass | c5f588f089d0 | bullet 1: pass; bullet 3: pass | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.llm-follows-switch | pass | c5f588f089d0 | bullet 4: pass | /sdk/S10-nokey/transcripts/S10-hooks-04.log | +| S10.matcher | pass | c5f588f089d0 | bullet 1: pass; bullet 3: pass | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.plugin-hooks-merge | pass | c5f588f089d0 | bullet 5: pass | /sdk/S10-nokey/transcripts/S10-hooks-05.log | +| S10.post-tool-use | pass | c5f588f089d0 | bullet 1: pass; bullet 6: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.pre-tool-use-block | pass | c5f588f089d0 | bullet 1: pass; bullet 6: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.prompt-hook | pass | c5f588f089d0 | bullet 4: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S10-nokey/transcripts/S10-hooks-04.log | +| S10.session-end-on-close | pass | c5f588f089d0 | bullet 1: pass | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.session-start | pass | c5f588f089d0 | bullet 1: pass | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.stop-hook | pass | c5f588f089d0 | bullet 1: pass; bullet 6: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 7: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S10.user-prompt-block | pass | c5f588f089d0 | bullet 2: pass | /sdk/S10-nokey/transcripts/S10-hooks-02.log | +| S10.user-prompt-context | pass | c5f588f089d0 | bullet 1: pass; bullet 6: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S10-nokey/transcripts/S10-hooks-01.log | +| S11.builtin-agents | pass | c5f588f089d0 | bullet 5: pass; bullet 7: pass; bullet 12: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S11-nokey/transcripts/S11-subagents-delegation-05.log | +| S11.conversation-autoregister | pass | c5f588f089d0 | bullet 7: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-07.log | +| S11.definition-factory | pass | c5f588f089d0 | bullet 4: pass; bullet 13: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S11-nokey/transcripts/S11-subagents-delegation-04.log | +| S11.definition-model | pass | c5f588f089d0 | bullet 4: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-04.log | +| S11.delegate-executor | pass | c5f588f089d0 | bullet 10: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-10.log | +| S11.delegation-visualizer | pass | c5f588f089d0 | bullet 10: pass; bullet 11: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S11-nokey/transcripts/S11-subagents-delegation-10.log | +| S11.file-discovery | pass | c5f588f089d0 | bullet 4: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-04.log | +| S11.file-precedence | pass | c5f588f089d0 | bullet 4: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-04.log | +| S11.markdown-schema | pass | c5f588f089d0 | bullet 4: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-04.log | +| S11.plugin-agents | pass | c5f588f089d0 | bullet 4: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-04.log | +| S11.register-if-absent | pass | c5f588f089d0 | bullet 4: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-04.log | +| S11.register-programmatic | pass | c5f588f089d0 | bullet 1: pass; bullet 4: pass; bullet 12: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S11-nokey/transcripts/S11-subagents-delegation-01.log | +| S11.sub-agent-scope | pass | c5f588f089d0 | bullet 9: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-09.log | +| S11.task-confirmation | pass | c5f588f089d0 | bullet 8: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-08.log | +| S11.task-errors | pass | c5f588f089d0 | bullet 1: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-01.log | +| S11.task-hooks | pass | c5f588f089d0 | bullet 1: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-01.log | +| S11.task-limits | pass | c5f588f089d0 | bullet 1: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-01.log | +| S11.task-metrics | pass | c5f588f089d0 | bullet 1: pass; bullet 12: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S11-nokey/transcripts/S11-subagents-delegation-01.log | +| S11.task-persistence | pass | c5f588f089d0 | bullet 1: pass; bullet 7: pass | /sdk/S11-nokey/transcripts/S11-subagents-delegation-01.log | +| S11.task-resume | pass | c5f588f089d0 | bullet 1: pass; bullet 11: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S11-nokey/transcripts/S11-subagents-delegation-01.log | +| S11.task-run | pass | c5f588f089d0 | bullet 1: pass; bullet 11: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S11-nokey/transcripts/S11-subagents-delegation-01.log | +| S12.api-evaluate | pass | c707d3737e8d | bullet 3: pass; bullet 5: blocked (bullet is labeled blocked) | /sdk/S12-nokey/transcripts/S12-critic-03.log | +| S12.api-followup-prompt | pass | c707d3737e8d | bullet 1: pass; bullet 3: pass | /sdk/S12-nokey/transcripts/S12-critic-01.log | +| S12.api-key-handling | pass | c707d3737e8d | bullet 1: pass | /sdk/S12-nokey/transcripts/S12-critic-01.log | +| S12.chat-template | pass | c707d3737e8d | bullet 1: pass | /sdk/S12-nokey/transcripts/S12-critic-01.log | +| S12.critic-display | pass | c707d3737e8d | bullet 1: pass; bullet 3: pass | /sdk/S12-nokey/transcripts/S12-critic-01.log | +| S12.critic-result | pass | c707d3737e8d | bullet 1: pass | /sdk/S12-nokey/transcripts/S12-critic-01.log | +| S12.custom-refine-policy | pass | c707d3737e8d | bullet 1: pass | /sdk/S12-nokey/transcripts/S12-critic-01.log | +| S12.end-to-end-refinement | pass | c707d3737e8d | bullet 3: pass; bullet 4: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 5: blocked (bullet is labeled blocked) | /sdk/S12-nokey/transcripts/S12-critic-03.log | +| S12.eval-mode | pass | c707d3737e8d | bullet 2: pass | /sdk/S12-nokey/transcripts/S12-critic-02.log | +| S12.refinement-config | pass | c707d3737e8d | bullet 1: pass | /sdk/S12-nokey/transcripts/S12-critic-01.log | +| S12.refinement-decision | pass | c707d3737e8d | bullet 2: pass | /sdk/S12-nokey/transcripts/S12-critic-02.log | +| S12.refinement-followup | pass | c707d3737e8d | bullet 2: pass | /sdk/S12-nokey/transcripts/S12-critic-02.log | +| S12.result-on-events | pass | c707d3737e8d | bullet 2: pass; bullet 4: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S12-nokey/transcripts/S12-critic-02.log | +| S12.rule-critics | pass | c707d3737e8d | bullet 1: pass | /sdk/S12-nokey/transcripts/S12-critic-01.log | +| S12.settings-critic | pass | c707d3737e8d | bullet 1: pass; bullet 3: pass | /sdk/S12-nokey/transcripts/S12-critic-01.log | +| S13.analysis-event | pass | c707d3737e8d | bullet 1: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-01.log | +| S13.confirm-approve-reject | pass | c707d3737e8d | bullet 1: pass; bullet 5: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 9: blocked (bullet is labeled blocked) | /sdk/S13-nokey/transcripts/S13-security-confirmation-01.log | +| S13.confirm-wait | pass | c707d3737e8d | bullet 1: pass; bullet 5: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S13-nokey/transcripts/S13-security-confirmation-01.log | +| S13.ensemble | pass | c707d3737e8d | bullet 2: pass; bullet 7: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-02.log | +| S13.grayswan | pass | c707d3737e8d | bullet 2: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-02.log | +| S13.llm-analyzer | pass | c707d3737e8d | bullet 1: pass; bullet 5: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 9: blocked (bullet is labeled blocked) | /sdk/S13-nokey/transcripts/S13-security-confirmation-01.log | +| S13.mask-async | pass | c707d3737e8d | bullet 4: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-04.log | +| S13.mask-models | pass | c707d3737e8d | bullet 3: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-03.log | +| S13.mask-output | pass | c707d3737e8d | bullet 3: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S13-nokey/transcripts/S13-security-confirmation-03.log | +| S13.mask-tool-output | pass | c707d3737e8d | bullet 3: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-03.log | +| S13.pattern-analyzer | pass | c707d3737e8d | bullet 2: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-02.log | +| S13.policies | pass | c707d3737e8d | bullet 2: pass; bullet 7: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-02.log | +| S13.policy-rails | pass | c707d3737e8d | bullet 2: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-02.log | +| S13.risk-extraction | pass | c707d3737e8d | bullet 1: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-01.log | +| S13.risk-levels | pass | c707d3737e8d | bullet 2: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-02.log | +| S13.secret-injection | pass | c707d3737e8d | bullet 3: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S13-nokey/transcripts/S13-security-confirmation-03.log | +| S13.secret-registry | pass | c707d3737e8d | bullet 3: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-03.log | +| S13.secret-sources | pass | c707d3737e8d | bullet 3: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-03.log | +| S13.security-policy-prompt | pass | c707d3737e8d | bullet 1: pass; bullet 6: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S13-nokey/transcripts/S13-security-confirmation-01.log | +| S13.settings-security | pass | c707d3737e8d | bullet 2: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-02.log | +| S13.toolshield | pass | c707d3737e8d | bullet 2: pass | /sdk/S13-nokey/transcripts/S13-security-confirmation-02.log | +| S14.apply-patch | pass | c707d3737e8d | bullet 1: pass; bullet 9: pass; bullet 11: pass; bullet 13: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 14: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S14-nokey/transcripts/S14-core-tools-01.log | +| S14.file-editor-edit | pass | c707d3737e8d | bullet 8: pass; bullet 12: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 14: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S14-nokey/transcripts/S14-core-tools-08.log | +| S14.file-editor-validation | pass | c707d3737e8d | bullet 8: pass | /sdk/S14-nokey/transcripts/S14-core-tools-08.log | +| S14.file-editor-view | pass | c707d3737e8d | bullet 8: pass; bullet 14: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S14-nokey/transcripts/S14-core-tools-08.log | +| S14.gemini-file-tools | pass | c707d3737e8d | bullet 11: pass; bullet 13: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 14: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S14-nokey/transcripts/S14-core-tools-11.log | +| S14.glob | pass | c707d3737e8d | bullet 1: pass; bullet 9: pass; bullet 10: pass | /sdk/S14-nokey/transcripts/S14-core-tools-01.log | +| S14.grep | pass | c707d3737e8d | bullet 1: pass; bullet 9: pass | /sdk/S14-nokey/transcripts/S14-core-tools-01.log | +| S14.preset-default | pass | c707d3737e8d | bullet 11: pass | /sdk/S14-nokey/transcripts/S14-core-tools-11.log | +| S14.preset-gpt5-gemini | pass | c707d3737e8d | bullet 11: pass; bullet 13: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S14-nokey/transcripts/S14-core-tools-11.log | +| S14.shared-working-dir | pass | c707d3737e8d | bullet 1: pass; bullet 12: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S14-nokey/transcripts/S14-core-tools-01.log | +| S14.terminal-cleanup | pass | c707d3737e8d | bullet 2: pass; bullet 3: pass | /sdk/S14-nokey/transcripts/S14-core-tools-02.log | +| S14.terminal-env | pass | c707d3737e8d | bullet 2: pass; bullet 3: pass | /sdk/S14-nokey/transcripts/S14-core-tools-02.log | +| S14.terminal-exec | pass | c707d3737e8d | bullet 1: pass; bullet 2: pass; bullet 3: pass; bullet 12: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 14: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S14-nokey/transcripts/S14-core-tools-01.log | +| S14.terminal-interactive | pass | c707d3737e8d | bullet 2: pass; bullet 4: pass; bullet 14: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S14-nokey/transcripts/S14-core-tools-02.log | +| S14.terminal-metadata | pass | c707d3737e8d | bullet 2: pass; bullet 3: pass | /sdk/S14-nokey/transcripts/S14-core-tools-02.log | +| S14.terminal-output-truncation | pass | c707d3737e8d | bullet 2: pass; bullet 3: pass | /sdk/S14-nokey/transcripts/S14-core-tools-02.log | +| S14.terminal-pane-pool | pass | c707d3737e8d | bullet 3: pass; bullet 4: pass | /sdk/S14-nokey/transcripts/S14-core-tools-03.log | +| S14.terminal-secret-masking | pass | c707d3737e8d | bullet 2: pass; bullet 3: pass | /sdk/S14-nokey/transcripts/S14-core-tools-02.log | +| S14.terminal-timeout | pass | c707d3737e8d | bullet 2: pass; bullet 3: pass | /sdk/S14-nokey/transcripts/S14-core-tools-02.log | +| S14.tool-output-masking | pass | c707d3737e8d | bullet 8: pass; bullet 9: pass | /sdk/S14-nokey/transcripts/S14-core-tools-08.log | +| S15.executor-config | pass | c707d3737e8d | bullet 4: pass | /sdk/S15-nokey/transcripts/S15-browser-04.log | +| S15.interact | pass | c707d3737e8d | bullet 1: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S15-nokey/transcripts/S15-browser-01.log | +| S15.navigate-and-read | pass | c707d3737e8d | bullet 1: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 9: blocked (bullet is labeled blocked) | /sdk/S15-nokey/transcripts/S15-browser-01.log | +| S15.observation | pass | c707d3737e8d | bullet 1: pass | /sdk/S15-nokey/transcripts/S15-browser-01.log | +| S15.preset-browser | pass | c707d3737e8d | bullet 7: pass | /sdk/S15-nokey/transcripts/S15-browser-07.log | +| S15.prompt-guidance | pass | c707d3737e8d | bullet 1: pass; bullet 2: blocked (block 1 is blocked (exit 3: a missing prerequisite)) | /sdk/S15-nokey/transcripts/S15-browser-01.log | +| S15.recording-start-stop | pass | c707d3737e8d | bullet 1: pass; bullet 6: blocked (bullet is labeled blocked); bullet 9: blocked (bullet is labeled blocked) | /sdk/S15-nokey/transcripts/S15-browser-01.log | +| S15.shared-executor | pass | c707d3737e8d | bullet 3: pass | /sdk/S15-nokey/transcripts/S15-browser-03.log | +| S15.storage | pass | c707d3737e8d | bullet 1: pass | /sdk/S15-nokey/transcripts/S15-browser-01.log | +| S15.tabs | pass | c707d3737e8d | bullet 1: pass | /sdk/S15-nokey/transcripts/S15-browser-01.log | +| S15.toolset-create | pass | c707d3737e8d | bullet 1: pass | /sdk/S15-nokey/transcripts/S15-browser-01.log | +| S16.oracle-consult | pass | c707d3737e8d | bullet 4: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-04.log | +| S16.oracle-errors | pass | c707d3737e8d | bullet 4: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-04.log | +| S16.oracle-tool | pass | c707d3737e8d | bullet 4: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-04.log | +| S16.plan-edit-restriction | pass | c707d3737e8d | bullet 2: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-02.log | +| S16.plan-file-headers | pass | c707d3737e8d | bullet 2: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-02.log | +| S16.plan-file-location | pass | c707d3737e8d | bullet 2: pass; bullet 3: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-02.log | +| S16.plan-then-execute | pass | c707d3737e8d | bullet 2: pass; bullet 7: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-02.log | +| S16.planning-agent | pass | c707d3737e8d | bullet 2: pass; bullet 7: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-02.log | +| S16.planning-tools | pass | c707d3737e8d | bullet 2: pass; bullet 3: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-02.log | +| S16.task-tracker-persistence | pass | c707d3737e8d | bullet 1: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-01.log | +| S16.task-tracker-plan-view | pass | c707d3737e8d | bullet 1: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-01.log | +| S16.tom-consult | pass | c707d3737e8d | bullet 5: pass; bullet 9: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-05.log | +| S16.tom-sleeptime | pass | c707d3737e8d | bullet 5: pass; bullet 9: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-05.log | +| S16.tom-storage-root | pass | c707d3737e8d | bullet 5: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-05.log | +| S16.workflow-errors-and-output | pass | c707d3737e8d | bullet 6: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-06.log | +| S16.workflow-fan-out | pass | c707d3737e8d | bullet 6: pass; bullet 10: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-06.log | +| S16.workflow-lifecycle | pass | c707d3737e8d | bullet 6: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-06.log | +| S16.workflow-pipeline | pass | c707d3737e8d | bullet 6: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-06.log | +| S16.workflow-script-validation | pass | c707d3737e8d | bullet 6: pass | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-06.log | +| S16.workflow-toolset | pass | c707d3737e8d | bullet 6: pass; bullet 10: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S16-nokey/transcripts/S16-planning-workflow-tools-06.log | +| S17.agent-tool-init | pass | c707d3737e8d | bullet 5: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-05.log | +| S17.annotations-and-resources | pass | c707d3737e8d | bullet 1: pass; bullet 3: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-01.log | +| S17.auto-naming | pass | c707d3737e8d | bullet 2: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-02.log | +| S17.auto-registration | pass | c707d3737e8d | bullet 2: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-02.log | +| S17.catalog | pass | c707d3737e8d | bullet 2: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-02.log | +| S17.client-tool-ack | pass | c707d3737e8d | bullet 6: pass; bullet 9: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-06.log | +| S17.client-tool-registration | pass | c707d3737e8d | bullet 7: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-07.log | +| S17.client-tool-schema | pass | c707d3737e8d | bullet 6: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-06.log | +| S17.client-tool-spec | pass | c707d3737e8d | bullet 7: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-07.log | +| S17.client-tools-conversation | pass | c707d3737e8d | bullet 6: pass; bullet 7: pass; bullet 9: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-06.log | +| S17.custom-tool-definition | pass | c707d3737e8d | bullet 1: pass; bullet 3: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-01.log | +| S17.execute-tool | pass | c707d3737e8d | bullet 1: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-01.log | +| S17.injected-fields | pass | c707d3737e8d | bullet 3: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-03.log | +| S17.llm-schemas | pass | c707d3737e8d | bullet 3: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-03.log | +| S17.module-qualnames | pass | c707d3737e8d | bullet 2: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-02.log | +| S17.output-coercion | pass | c707d3737e8d | bullet 3: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-03.log | +| S17.py-type | pass | c707d3737e8d | bullet 3: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-03.log | +| S17.register-tool | pass | c707d3737e8d | bullet 2: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-02.log | +| S17.resolve-tool | pass | c707d3737e8d | bullet 1: pass; bullet 2: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-01.log | +| S17.schema-serialization | pass | c707d3737e8d | bullet 3: pass; bullet 4: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-03.log | +| S17.tool-set | pass | c707d3737e8d | bullet 1: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-01.log | +| S17.tool-spec | pass | c707d3737e8d | bullet 1: pass; bullet 2: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-01.log | +| S17.usability | pass | c707d3737e8d | bullet 2: pass | /sdk/S17-nokey/transcripts/S17-custom-and-client-tools-02.log | +| S18.async-remote | pass | 1ecd2de9dc6b | bullet 2: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-02.log | +| S18.cloud-automation-tags | pass | 1ecd2de9dc6b | bullet 7: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-07.log | +| S18.cloud-credentials | pass | 1ecd2de9dc6b | bullet 7: pass; bullet 10: blocked (bullet is labeled blocked) | /sdk/S18-nokey/transcripts/S18-workspace-backends-07.log | +| S18.cloud-local-mode | pass | 1ecd2de9dc6b | bullet 7: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-07.log | +| S18.completion-callback | pass | 1ecd2de9dc6b | bullet 1: pass; bullet 5: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-01.log | +| S18.conversation-binding | pass | 1ecd2de9dc6b | bullet 1: pass; bullet 5: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S18-nokey/transcripts/S18-workspace-backends-01.log | +| S18.local-ops | pass | 1ecd2de9dc6b | bullet 1: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-01.log | +| S18.local-pause-resume | pass | 1ecd2de9dc6b | bullet 1: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-01.log | +| S18.remote-alive | pass | 1ecd2de9dc6b | bullet 2: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-02.log | +| S18.remote-client | pass | 1ecd2de9dc6b | bullet 2: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-02.log | +| S18.remote-command-exec | pass | 1ecd2de9dc6b | bullet 2: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S18-nokey/transcripts/S18-workspace-backends-02.log | +| S18.remote-files-git | pass | 1ecd2de9dc6b | bullet 2: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-02.log | +| S18.remote-settings | pass | 1ecd2de9dc6b | bullet 4: pass; bullet 8: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S18-nokey/transcripts/S18-workspace-backends-04.log | +| S18.repos-and-skills | pass | 1ecd2de9dc6b | bullet 6: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-06.log | +| S18.runtime-scope | pass | 1ecd2de9dc6b | bullet 5: pass | /sdk/S18-nokey/transcripts/S18-workspace-backends-05.log | +| S19.acp-providers-catalog | pass | 5ff99f1b6ca2 | bullet 4: pass; bullet 5: pass | /sdk/S19/transcripts/S19-settings-04.log | +| S19.acp-settings | pass | 5ff99f1b6ca2 | bullet 2: pass | /sdk/S19/transcripts/S19-settings-02.log | +| S19.apply-diff | pass | 5ff99f1b6ca2 | bullet 2: pass | /sdk/S19/transcripts/S19-settings-02.log | +| S19.condenser-settings | pass | 5ff99f1b6ca2 | bullet 1: pass; bullet 10: pass | /sdk/S19/transcripts/S19-settings-01.log | +| S19.conversation-settings | pass | 5ff99f1b6ca2 | bullet 2: pass | /sdk/S19/transcripts/S19-settings-02.log | +| S19.credential-binding | pass | 5ff99f1b6ca2 | bullet 9: pass | /sdk/S19/transcripts/S19-settings-09.log | +| S19.mcp-config-secrets | pass | 5ff99f1b6ca2 | bullet 1: pass | /sdk/S19/transcripts/S19-settings-01.log | +| S19.persisted-baselines | pass | 5ff99f1b6ca2 | bullet 6: pass; bullet 7: pass | /sdk/S19/transcripts/S19-settings-06.log | +| S19.profile-migrations | pass | 5ff99f1b6ca2 | bullet 8: pass | /sdk/S19/transcripts/S19-settings-08.log | +| S19.profile-model | pass | 5ff99f1b6ca2 | bullet 8: pass | /sdk/S19/transcripts/S19-settings-08.log | +| S19.profile-refs | pass | 5ff99f1b6ca2 | bullet 8: pass | /sdk/S19/transcripts/S19-settings-08.log | +| S19.profile-resolve | pass | 5ff99f1b6ca2 | bullet 8: pass | /sdk/S19/transcripts/S19-settings-08.log | +| S19.profile-seed | pass | 5ff99f1b6ca2 | bullet 8: pass | /sdk/S19/transcripts/S19-settings-08.log | +| S19.profile-store | pass | 5ff99f1b6ca2 | bullet 8: pass | /sdk/S19/transcripts/S19-settings-08.log | +| S19.pypi-baseline-gate | pass | 5ff99f1b6ca2 | bullet 7: pass | /sdk/S19/transcripts/S19-settings-07.log | +| S19.retired-switches | pass | 5ff99f1b6ca2 | bullet 1: pass | /sdk/S19/transcripts/S19-settings-01.log | +| S19.schema-export | pass | 5ff99f1b6ca2 | bullet 2: pass | /sdk/S19/transcripts/S19-settings-02.log | +| S19.schema-migrations | pass | 5ff99f1b6ca2 | bullet 2: pass | /sdk/S19/transcripts/S19-settings-02.log | +| S19.settings-create-agent | pass | 5ff99f1b6ca2 | bullet 1: pass; bullet 10: pass | /sdk/S19/transcripts/S19-settings-01.log | +| S19.settings-roundtrip | pass | 5ff99f1b6ca2 | bullet 1: pass; bullet 10: pass | /sdk/S19/transcripts/S19-settings-01.log | +| S19.settings-tools | pass | 5ff99f1b6ca2 | bullet 1: pass; bullet 10: pass | /sdk/S19/transcripts/S19-settings-01.log | +| S19.subscription-llm-settings | pass | 5ff99f1b6ca2 | bullet 2: pass; bullet 3: blocked (bullet is labeled blocked) | /sdk/S19/transcripts/S19-settings-02.log | +| S19.tool-concurrency-persona | pass | 5ff99f1b6ca2 | bullet 1: pass | /sdk/S19/transcripts/S19-settings-01.log | +| S19.validate-dispatch | pass | 5ff99f1b6ca2 | bullet 2: pass | /sdk/S19/transcripts/S19-settings-02.log | +| S19.verification-settings | pass | 5ff99f1b6ca2 | bullet 1: pass | /sdk/S19/transcripts/S19-settings-01.log | +| S20.automation-context | pass | 0da9cf850a0e | bullet 7: pass | /sdk/S20-nokey/transcripts/S20-observability-07.log | +| S20.automation-headers | pass | 0da9cf850a0e | bullet 7: pass | /sdk/S20-nokey/transcripts/S20-observability-07.log | +| S20.automation-kwargs | pass | 0da9cf850a0e | bullet 7: pass | /sdk/S20-nokey/transcripts/S20-observability-07.log | +| S20.conversation-args | pass | 0da9cf850a0e | bullet 1: pass | /sdk/S20-nokey/transcripts/S20-observability-01.log | +| S20.delegate-trace | pass | 0da9cf850a0e | bullet 5: pass | /sdk/S20-nokey/transcripts/S20-observability-05.log | +| S20.enable-gate | pass | 0da9cf850a0e | bullet 1: pass; bullet 3: pass; bullet 6: pass; bullet 7: pass | /sdk/S20-nokey/transcripts/S20-observability-01.log | +| S20.env-conversation-defaults | pass | 0da9cf850a0e | bullet 2: pass | /sdk/S20-nokey/transcripts/S20-observability-02.log | +| S20.external-entry | pass | 0da9cf850a0e | bullet 2: pass; bullet 6: pass | /sdk/S20-nokey/transcripts/S20-observability-02.log | +| S20.get-env | pass | 0da9cf850a0e | bullet 7: pass | /sdk/S20-nokey/transcripts/S20-observability-07.log | +| S20.laminar-init | pass | 0da9cf850a0e | bullet 3: pass; bullet 10: blocked (bullet is labeled blocked) | /sdk/S20-nokey/transcripts/S20-observability-03.log | +| S20.lazy-import | pass | 0da9cf850a0e | bullet 6: pass | /sdk/S20-nokey/transcripts/S20-observability-06.log | +| S20.llm-span | pass | 0da9cf850a0e | bullet 4: pass; bullet 9: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S20-nokey/transcripts/S20-observability-04.log | +| S20.logger-config | pass | 0da9cf850a0e | bullet 8: pass | /sdk/S20-nokey/transcripts/S20-observability-08.log | +| S20.logger-quiet-thirdparty | pass | 0da9cf850a0e | bullet 8: pass | /sdk/S20-nokey/transcripts/S20-observability-08.log | +| S20.otel-backend | pass | 0da9cf850a0e | bullet 1: pass | /sdk/S20-nokey/transcripts/S20-observability-01.log | +| S20.parallel-span-context | pass | 0da9cf850a0e | bullet 1: pass | /sdk/S20-nokey/transcripts/S20-observability-01.log | +| S20.rolling-log-view | pass | 0da9cf850a0e | bullet 8: pass | /sdk/S20-nokey/transcripts/S20-observability-08.log | +| S20.root-span | pass | 0da9cf850a0e | bullet 1: pass; bullet 9: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)); bullet 10: blocked (bullet is labeled blocked) | /sdk/S20-nokey/transcripts/S20-observability-01.log | +| S20.step-and-tool-spans | pass | 0da9cf850a0e | bullet 1: pass; bullet 9: blocked (1 model-backed block(s) skipped (no DEEPSEEK_API_KEY)) | /sdk/S20-nokey/transcripts/S20-observability-01.log | +| S20.utility-operation-metadata | pass | 0da9cf850a0e | bullet 4: pass | /sdk/S20-nokey/transcripts/S20-observability-04.log | +| S21.auth-selection | pass | 07e3bb94b792 | bullet 8: pass | /sdk/S21-effort/transcripts/S21-acp-agent-08.log | +| S21.cancel-pause | pass | 07e3bb94b792 | bullet 7: pass | /sdk/S21-effort/transcripts/S21-acp-agent-07.log | +| S21.cleanup | pass | 07e3bb94b792 | bullet 1: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.construct | pass | 07e3bb94b792 | bullet 1: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.effort | pass | 07e3bb94b792 | bullet 10: pass | /sdk/S21-effort/transcripts/S21-acp-agent-10.log | +| S21.effort-switch | pass | 07e3bb94b792 | bullet 10: pass | /sdk/S21-effort/transcripts/S21-acp-agent-10.log | +| S21.file-secrets | pass | 07e3bb94b792 | bullet 8: pass | /sdk/S21-effort/transcripts/S21-acp-agent-08.log | +| S21.handshake-live | pass | 07e3bb94b792 | bullet 13: pass | /sdk/S21-effort/transcripts/S21-acp-agent-13.log | +| S21.init-state | pass | 07e3bb94b792 | bullet 1: pass; bullet 4: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.masking | pass | 07e3bb94b792 | bullet 1: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.mcp-forwarding | pass | 07e3bb94b792 | bullet 1: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.model-discovery | pass | 07e3bb94b792 | bullet 9: pass; bullet 10: pass | /sdk/S21-effort/transcripts/S21-acp-agent-09.log | +| S21.model-selection | pass | 07e3bb94b792 | bullet 1: pass; bullet 3: pass; bullet 8: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.model-switch | pass | 07e3bb94b792 | bullet 1: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.permissions-auto-approve | pass | 07e3bb94b792 | bullet 1: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.provider-registry | pass | 07e3bb94b792 | bullet 12: pass; bullet 13: pass | /sdk/S21-effort/transcripts/S21-acp-agent-12.log | +| S21.secrets-env | pass | 07e3bb94b792 | bullet 1: pass; bullet 8: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.session-resume | pass | 07e3bb94b792 | bullet 3: pass | /sdk/S21-effort/transcripts/S21-acp-agent-03.log | +| S21.settings | pass | 07e3bb94b792 | bullet 9: pass; bullet 10: pass | /sdk/S21-effort/transcripts/S21-acp-agent-09.log | +| S21.step | pass | 07e3bb94b792 | bullet 1: pass; bullet 14: blocked (bullet is labeled blocked) | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.tool-call-events | pass | 07e3bb94b792 | bullet 1: pass | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | +| S21.tracing | pass | 07e3bb94b792 | bullet 11: pass | /sdk/S21-effort/transcripts/S21-acp-agent-11.log | +| S21.usage-cost | pass | 07e3bb94b792 | bullet 1: pass; bullet 14: blocked (bullet is labeled blocked) | /sdk/S21-effort/transcripts/S21-acp-agent-01.log | diff --git a/.pr/map-maintenance/2026-11-11/ui/F05.queued-message-answered.png b/.pr/map-maintenance/2026-11-11/ui/F05.queued-message-answered.png new file mode 100644 index 00000000000..a0753e859fe Binary files /dev/null and b/.pr/map-maintenance/2026-11-11/ui/F05.queued-message-answered.png differ diff --git a/.pr/map-maintenance/2026-11-11/ui/F05.queued-message-messages.json b/.pr/map-maintenance/2026-11-11/ui/F05.queued-message-messages.json new file mode 100644 index 00000000000..62db57d1f8e --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/ui/F05.queued-message-messages.json @@ -0,0 +1,27 @@ +{ + "ok": true, + "total": 13, + "pages": 1, + "more": false, + "count": 3, + "events": [ + { + "kind": "MessageEvent", + "source": "user", + "text": "Run this shell command: sleep 25 && echo slept. Then reply with only: done", + "ts": "2026-11-11T08:55:58.637129" + }, + { + "kind": "MessageEvent", + "source": "user", + "text": "Also reply with: queued-ok", + "ts": "2026-11-11T08:56:26.300042" + }, + { + "kind": "MessageEvent", + "source": "agent", + "text": "done queued-ok", + "ts": "2026-11-11T08:56:28.811768" + } + ] +} diff --git a/.pr/map-maintenance/2026-11-11/ui/F05.queued-message-sent-while-running.png b/.pr/map-maintenance/2026-11-11/ui/F05.queued-message-sent-while-running.png new file mode 100644 index 00000000000..50f3aaca873 Binary files /dev/null and b/.pr/map-maintenance/2026-11-11/ui/F05.queued-message-sent-while-running.png differ diff --git a/.pr/map-maintenance/2026-11-11/ui/F07.download-zip-trajectory-overlap-summary.json b/.pr/map-maintenance/2026-11-11/ui/F07.download-zip-trajectory-overlap-summary.json new file mode 100644 index 00000000000..ef7a9ee8ce1 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/ui/F07.download-zip-trajectory-overlap-summary.json @@ -0,0 +1,19 @@ +{ + "origin": "http://127.0.0.1:18800", + "clients": 3, + "duration_limit_seconds": 20.0, + "round_limit_per_client": 5000, + "slow_ms": 2.0, + "elapsed_seconds": 20.067, + "max_network_bytes": 16777216, + "max_uncompressed_bytes": 16777216, + "max_members": 512, + "counts": { + "attempts": 733, + "http_200": 733, + "valid_zip": 730, + "received_bytes": 84115939, + "unverified_deadline": 3 + }, + "raw_samples_private": true +} diff --git a/.pr/map-maintenance/2026-11-11/ui/F07.download-zip-ui-download-inspect.json b/.pr/map-maintenance/2026-11-11/ui/F07.download-zip-ui-download-inspect.json new file mode 100644 index 00000000000..4ce2d730496 --- /dev/null +++ b/.pr/map-maintenance/2026-11-11/ui/F07.download-zip-ui-download-inspect.json @@ -0,0 +1,50 @@ +{ + "ok": true, + "downloads": [ + { + "ts": "2026-11-11T08:56:47.784Z", + "kind": "download", + "page": "http://127.0.0.1:18800/conversations/69779bf0-be6a-42f9-9e6b-4a368a05686a", + "appOrigin": true, + "path": "/oh/2026-11-11T0853-78d182/private/downloads/1794387407781-conversation_69779bf0-be6a-42f9-9e6b-4a368a05686a.zip", + "inspect": { + "bytes": 115329, + "zip": { + "count": 38, + "names": [ + "69779bf0be6a42f99e6b4a368a05686a/", + "69779bf0be6a42f99e6b4a368a05686a/.owner_lease.lock", + "69779bf0be6a42f99e6b4a368a05686a/base_state.json", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085559368651_BashCommand_5318a5cb44884ec4aa551ae20542c082", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085559380137_BashCommand_f1ebe10d7e274b578a854334723022b6", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085559400870_BashOutput_5318a5cb44884ec4aa551ae20542c082_6c2ae1823ef0482ab1f8c3675dee82e5", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085559412030_BashOutput_f1ebe10d7e274b578a854334723022b6_47306cd1b0a64a86a1c9e1b8ae7144dc", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085609494406_BashCommand_86622db49aa74581a1a55a171911e59f", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085609502651_BashOutput_86622db49aa74581a1a55a171911e59f_7fb53e09c7454ffdb61a93597c487113", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085619535253_BashCommand_62ce6b757eb445cc8a87153d49c75753", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085619564078_BashOutput_62ce6b757eb445cc8a87153d49c75753_d2933e8ff3b5442a8da1e0f14df4d4a5", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085629589432_BashCommand_902d732ec8f448c6869b048a46aa8645", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085629609603_BashOutput_902d732ec8f448c6869b048a46aa8645_18a2566fc97b4cbb863d47694a783ac2", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085639640965_BashCommand_7db940156e3d47f2b71a68c8640c26c2", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085639661772_BashOutput_7db940156e3d47f2b71a68c8640c26c2_72f68eebfdcf45d78865e11442221ad3", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085646089043_BashCommand_91687ef4c82b45ceb147ed48842b162c", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085646089911_BashCommand_689844e67d464b91a09856f78c18bed4", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085646143656_BashOutput_91687ef4c82b45ceb147ed48842b162c_4a1424cb44a5422e81555b96f99e3417", + "69779bf0be6a42f99e6b4a368a05686a/bash_events/20261111085646146494_BashOutput_689844e67d464b91a09856f78c18bed4_90e5bb8442134932a0dca4184ba63fe7", + "69779bf0be6a42f99e6b4a368a05686a/events/", + "69779bf0be6a42f99e6b4a368a05686a/events/.eventlog-len-13.marker", + "69779bf0be6a42f99e6b4a368a05686a/events/.eventlog.lock", + "69779bf0be6a42f99e6b4a368a05686a/events/event-00000-5b939461-bf97-462a-881d-ebda44c66bb2.json", + "69779bf0be6a42f99e6b4a368a05686a/events/event-00001-0f3bcc3a-8411-4156-805b-37df48c708f2.json", + "69779bf0be6a42f99e6b4a368a05686a/events/event-00002-ffc0f510-d5f7-420a-8d70-e11476e9ab75.json", + "69779bf0be6a42f99e6b4a368a05686a/events/event-00003-7a47a4b1-268f-4561-ad50-4eda750926b0.json", + "69779bf0be6a42f99e6b4a368a05686a/events/event-00004-17af1a17-2013-4717-ae3f-b0204950112d.json", + "69779bf0be6a42f99e6b4a368a05686a/events/event-00005-e336100b-46a0-4d96-b6ff-27a3a095b0f9.json", + "69779bf0be6a42f99e6b4a368a05686a/events/event-00006-71d329e2-613a-4468-bd0c-0aabccc60c38.json" + ] + } + } + } + ] +}