diff --git a/.agents/skills/verify-agent-server/references/feature-map/F03-deferred-init.md b/.agents/skills/verify-agent-server/references/feature-map/F03-deferred-init.md index 5da35c0f7f8..e2ec404731d 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F03-deferred-init.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F03-deferred-init.md @@ -100,10 +100,12 @@ Routes: `GET /api/init`, `POST /api/init` capacity), and a CORS preflight `OPTIONS /api/settings` with an `Origin` (CORS allowlist); the delivered webhook's deliveries are read from an HTTP sink. -- SDK / TypeScript client: no consumer method calls `/api/init`; only the - generated TypeScript types describe it - (`clients/typescript/src/generated/agent-server-schema.ts`). The caller is a - warm-pool orchestrator. +- TypeScript client: `InitClient.getStatus()` (`GET /api/init`) and + `InitClient.initialize(request, {initApiKey})` (`POST /api/init`, sending + `initApiKey` as `X-Init-API-Key`), also reachable as + `AgentServerClient.init` and `ConversationManager.init`; driven against a + dormant server by `F34.init-client`. The Python SDK has no method for + `/api/init`. The usual caller is a warm-pool orchestrator. ## Driving it with control-agent-server diff --git a/.agents/skills/verify-agent-server/references/feature-map/F06-conversation-events.md b/.agents/skills/verify-agent-server/references/feature-map/F06-conversation-events.md index e11580c181b..548fd7a0dfa 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F06-conversation-events.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F06-conversation-events.md @@ -5,7 +5,8 @@ user and agent message, every tool action and observation, and one state update per changed field (`execution_status`, `stats`, `last_user_message_id`, ...). Consumers append a user message with `POST .../events` (and optionally start the agent), read history through search (filters, cursor paging, sort), -count, get-one and batch-get, and follow the conversation live over +count, get-one and batch-get (a GET with a JSON body, or its browser-safe +`POST .../events/batch` form), and follow the conversation live over `/sockets/events/{conversation_id}`: first-frame auth, a transient `full_state` snapshot, optional replay of history (`resend_mode=all` or `since`), then every new event as raw JSON; a message frame sent on the socket @@ -21,6 +22,7 @@ Routes: `GET /api/conversations/{conversation_id}/events/search`, `GET /api/conversations/{conversation_id}/events/count`, `GET /api/conversations/{conversation_id}/events/{event_id}`, `GET /api/conversations/{conversation_id}/events`, +`POST /api/conversations/{conversation_id}/events/batch`, `POST /api/conversations/{conversation_id}/events`, `WS /sockets/events/{conversation_id}` @@ -36,6 +38,7 @@ Routes: `GET /api/conversations/{conversation_id}/events/search`, - `F06.get-one`: `GET .../events/{event_id}` returns the event that search lists under that id, and the same event is on disk. - `F06.unknown-event`: an unknown event id returns 404 from get-one and `null` in its slot from batch-get. - `F06.batch-get`: `GET .../events` with a JSON array of ids as the body returns those events in request order; no body is a 422. +- `F06.batch-post`: `POST .../events/batch` with the same JSON array returns byte for byte what the GET batch returns (events in request order, `null` for an unknown id, `[]` for `[]`), appends nothing, and answers 422 for no body or a non-array body, 404 for an unknown conversation and 401 without a key. - `F06.not-found-auth`: every event route returns 404 for an unknown conversation, 422 for a malformed conversation id and 401 without a valid session key. - `F06.send-validation`: a missing body, a message with an unknown role, string content or an unknown content type is rejected with 422 and nothing is appended. - `F06.send-image`: a message with a text part and an `image` part (a `data:` URL) is persisted with its `image_urls` intact and returned the same by search, get-one and the socket replay; an `image` part without `image_urls` is a 422 and appends nothing. @@ -81,8 +84,10 @@ Routes: `GET /api/conversations/{conversation_id}/events/search`, - REST: `GET .../events/{event_id}` (TypeScript `ConversationClient.getEvent()`, `RemoteEventsList.getEventById()`) and `GET .../events` with a JSON array body (batch get; no SDK or TypeScript caller, because browsers cannot send a GET - body: the TypeScript `getEvents()` fans out to get-one and maps 404 to - `null`). + body) and its browser-safe form `POST .../events/batch` with the same bare + JSON array body (TypeScript `ConversationClient.batchGetEvents()`, driven by + `F34.events-batch-post`). The TypeScript `RemoteEventsList.getEventsById()` + still fans out to get-one and maps 404 to `null`. - WebSocket: `/sockets/events/{conversation_id}` with query `resend_mode` (`all` or `since`), `after_timestamp` (ISO 8601, required for `since`) and the deprecated `resend_all=true` and `session_api_key`. The SDK's @@ -280,6 +285,35 @@ Preconditions: ``` Events come back in request order, not log order; without a body the route answers 422 `body: Field required`. +- **Batch get by POST (`F06.batch-post`).** The same ids through the + browser-safe form, compared byte for byte with the GET form. + + Requires: `F06.search-time-window`, `F06.get-one` + ```sh + B6="$AGENT_SERVER_VERIFY_RUN/fixtures/qa-f06-batch" + mkdir -p "$B6" + N6=$(control-agent-server api GET "/api/conversations/$CID/events/count" --field .) + control-agent-server api POST "/api/conversations/$CID/events/batch" --json "[\"$E2\", \"qa-missing-event\", \"$E1\"]" --expect 200 \ + --check . len-eq 3 --check 0.id eq "$E2" --check 1 missing --check 2.id eq "$E1" --raw-out "$B6/post.json" --save F06.batch-post/three + control-agent-server api GET "/api/conversations/$CID/events" --json "[\"$E2\", \"qa-missing-event\", \"$E1\"]" --expect 200 \ + --quiet --raw-out "$B6/get.json" + cmp "$B6/get.json" "$B6/post.json" + control-agent-server api POST "/api/conversations/$CID/events/batch" --json '[]' --expect 200 --check . len-eq 0 + control-agent-server api POST "/api/conversations/$CID/events/batch" --expect 422 --check detail.0.loc contains body \ + --check detail.0.type eq missing --save F06.batch-post/no-body + control-agent-server api POST "/api/conversations/$CID/events/batch" --json "{\"ids\": [\"$E2\"]}" --expect 422 \ + --check detail.0.type eq list_type --save F06.batch-post/object-body + control-agent-server api POST "/api/conversations/$QA_UNKNOWN/events/batch" --json "[\"$E2\"]" --expect 404 \ + --check detail contains 'Conversation not found' --save F06.batch-post/unknown-conversation + control-agent-server api POST "/api/conversations/$CID/events/batch" --json "[\"$E2\"]" --auth none --expect 401 + control-agent-server api POST "/api/conversations/$CID/events/batch" --json "[\"$E2\"]" --expect 200 --check 0.id eq "$E2" + control-agent-server api GET "/api/conversations/$CID/events/count" --check . eq "$N6" + ``` + The POST answers `[, null, ]`, the same bytes as the GET form + for the same ids; `[]` is `[]`. No body is 422 `missing` at `body`, an + object instead of the bare array is 422 `list_type`, an unknown + conversation 404 `Conversation not found`, no key 401, and the count is + unchanged: the POST reads, it appends nothing. - **Unknown conversation, bad id, no key (`F06.not-found-auth`).** Requires: `F06.get-one` @@ -766,7 +800,9 @@ Preconditions: them. Compare ids, not raw JSON. - An unknown event id is a 404, and `null` in its slot of a batch (`F06.unknown-event`); batch-get is a GET with a JSON body, which browsers - cannot send. + cannot send, so browser clients use `POST .../events/batch` + (`F06.batch-post`). `GET .../events/batch` is not that route: it is + get-one for the event id `batch` (404). - The `SystemPromptEvent` is written on the first message, not at creation; a conversation created without a message has an empty log and count 0. - `POST .../events` with `run: false` never runs, even on an idle diff --git a/.agents/skills/verify-agent-server/references/feature-map/F07-session-socket.md b/.agents/skills/verify-agent-server/references/feature-map/F07-session-socket.md index 0f857937ad7..b1a46de0712 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F07-session-socket.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F07-session-socket.md @@ -27,6 +27,7 @@ Routes: `WS /sockets/session/{conversation_id}` - `F07.sync-first-during-run`: while a run holds the conversation's state lock, `sync` is still the first frame; an `error` reply to an invalid frame sent right after connecting comes after it. - `F07.snapshot-transient`: every connection gets one `transient` frame with a `full_state` `ConversationStateUpdateEvent` and no `seq`, after `sync` (and after the replay when one ran); it is never persisted. - `F07.snapshot-compact-stats`: the `full_state` snapshot carries each LLM's aggregate stats (`accumulated_cost`, `accumulated_token_usage`) but not the per-call `costs`, `response_latencies` and `token_usages` lists, so it stays small however long the conversation runs; `GET /api/conversations/{id}` returns the same aggregates with empty per-call lists. +- `F07.snapshot-no-skill-bodies`: the `full_state` snapshot and live `agent` state updates carry the agent with `agent_context.skills` emptied, so skill bodies never ride the socket live, while `GET /api/conversations/{id}?include_skills=true` and the stored event keep them. - `F07.full-replay`: `after_seq=-1` (or any value below it) replays every persisted event as `durable` frames with `seq` 0..`through_seq`, in order, once each, with the same ids REST returns. - `F07.seq-on-disk`: a durable frame's `seq` is the event's on-disk index, the `NNNNN` in `events/event-NNNNN-.json`. - `F07.resume-cursor`: `after_seq=k` replays exactly the events with `seq > k`; `after_seq=through_seq` replays nothing. @@ -479,6 +480,41 @@ Preconditions: conversation's `base_state.json` still keeps every per-call entry; only the wire payloads are compact. The legacy events socket's snapshot (F06) is built the same way. +- **Snapshot without skill bodies (`F07.snapshot-no-skill-bodies`).** A + conversation whose agent carries an inline skill (a placeholder model, never + run): read its live-only snapshot, then switch its LLM (F09's route) while a + capture is open, and compare with the stored state. + ```sh + KLLM='"llm": {"model": "openai/qa-placeholder", "api_key": "qa-placeholder", "base_url": "http://127.0.0.1:9/v1", "num_retries": 0}' + KID=$(control-agent-server conversation start --no-autotitle --print-id --body-json \ + '{"agent_settings": {"agent_kind": "openhands", '"$KLLM"', "tools": [], "agent_context": {"skills": [{"name": "qa-f07-skill", "content": "QA_F07_SKILL_BODY", "trigger": null}]}}}') + control-agent-server api GET "/api/conversations/$KID" --query include_skills=true --expect 200 \ + --check agent.agent_context.skills.0.name eq qa-f07-skill --check agent.agent_context.skills.0.content eq QA_F07_SKILL_BODY + control-agent-server ws listen "/sockets/session/$KID" --count 2 --save "F07.snapshot-no-skill-bodies/live-$KID" + control-agent-server state cat "evidence/F07.snapshot-no-skill-bodies/live-$KID.json" --not-contains QA_F07_SKILL_BODY \ + --check frames.2.frame.event.key eq full_state --check frames.2.frame.event.value.agent.agent_context exists \ + --check frames.2.frame.event.value.agent.agent_context.skills len-eq 0 + control-agent-server ws start "/sockets/session/$KID" --name "qa-f07-skills-$KID" + control-agent-server ws read "qa-f07-skills-$KID" --wait 10 --contains '"key": "full_state"' --expect-min 1 + control-agent-server api POST "/api/conversations/$KID/switch_llm" --expect 200 --check success eq true \ + --json '{"llm": {"model": "openai/qa-placeholder-2", "api_key": "qa-placeholder", "base_url": "http://127.0.0.1:9/v1", "num_retries": 0, "usage_id": "qa-f07-switched"}}' + control-agent-server ws stop "qa-f07-skills-$KID" --wait 10 --kinds durable:ConversationStateUpdateEvent --contains '"key": "agent"' \ + --expect-min 1 --save "F07.snapshot-no-skill-bodies/agent-$KID" + control-agent-server state cat "evidence/F07.snapshot-no-skill-bodies/agent-$KID.json" --not-contains QA_F07_SKILL_BODY \ + --check frames.3.frame.type eq durable --check frames.3.frame.event.key eq agent \ + --check frames.3.frame.event.value.llm.model eq openai/qa-placeholder-2 --check frames.3.frame.event.value.agent_context.skills len-eq 0 + control-agent-server api GET "/api/conversations/$KID" --query include_skills=true --expect 200 \ + --check agent.llm.model eq openai/qa-placeholder-2 --check agent.agent_context.skills.0.content eq QA_F07_SKILL_BODY \ + --save "F07.snapshot-no-skill-bodies/info-$KID" + control-agent-server api GET "/api/conversations/$KID/events/search" --expect 200 --check items len-eq 1 \ + --check items.0.key eq agent --check items.0.value.agent_context.skills.0.content eq QA_F07_SKILL_BODY + ``` + The conversation holds the skill (REST with `include_skills=true`), yet + the snapshot's agent has `agent_context.skills: []` and no frame of either + capture contains the body. The `switch_llm` publishes the new agent as a + durable `agent` update (seq 0, the conversation's only event), whose live + frame also has an empty skill list next to the new model. Nothing is lost: + the conversation and the stored event keep the skill with its body. - **Interrupted stream (`F07.item-aborted`).** Start a streamed deepseek-flash answer that takes tens of seconds (counting to 3000), wait until its `delta` frames flow on a background capture, then interrupt the @@ -791,4 +827,9 @@ Preconditions: POSTs above use `--quiet` to keep the map-run transcript small (`--expect` and `--check` still see the bodies); `--max-chars` only trims non-JSON response bodies. +- Only frames published live are trimmed of skill bodies + (`F07.snapshot-no-skill-bodies`): a replay (`after_seq`) reads the stored + event, so the same durable `agent` update at the same `seq` carries + `agent_context.skills` with their bodies, as `GET .../events/search` does. + A client that needs the skills reads `GET /api/conversations/{id}?include_skills=true`. - Agent Canvas bundles Agent Server 1.62.0 by default, whose `full_state` snapshot still carries the per-call lists, so a very long conversation keeps closing with 1013 `frame_too_large` in Canvas until it pins a release that has `F07.snapshot-compact-stats` ([OpenHands/OpenHands#18935](https://github.com/OpenHands/OpenHands/issues/18935)). diff --git a/.agents/skills/verify-agent-server/references/feature-map/F09-model-switching-plugins.md b/.agents/skills/verify-agent-server/references/feature-map/F09-model-switching-plugins.md index 072192c19dc..583071b78b6 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F09-model-switching-plugins.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F09-model-switching-plugins.md @@ -55,7 +55,7 @@ Routes: `POST /api/conversations/{conversation_id}/switch_profile`, - `F09.ts-client`: the TypeScript client's `ConversationClient.switchLLM`, `switchProfile` and `switchAcpModel` and `ConversationManager.switchProfile` reach the routes: each switch resolves and reads back through `getConversation`, an unknown profile rejects with 404, `switchAcpModel` on a regular conversation with 400, a call without a key with 401, and `switchAcpModel` on an ACP conversation that has not run resolves and stores the model. - `F09.ts-client-acp-docs`: the TypeScript client's `switchAcpModel` docstrings describe what the server does before an ACP conversation's first run (200, the model is stored for the first session), not a 409. - `F09.ts-client-acp-effort`: the TypeScript client's `ConversationClient.switchAcpEffort` stores the level on an ACP conversation that has not run (`getConversation` shows `acp_effort` and `current_effort`), and rejects with 400 on a regular conversation and with 401 without a key. -- `F09.load-plugin`: `load_plugin` with `@` from a local marketplace registered on the agent's `agent_context` answers 200 and merges the plugin's skill, its command and its hooks into the live agent, visible in `GET`, `base_state.json` and on the socket. +- `F09.load-plugin`: `load_plugin` with `@` from a local marketplace registered on the agent's `agent_context` answers 200 and merges the plugin's skill, its command and its hooks into the live agent, visible in `GET` and `base_state.json`; the socket pushes the `agent` and `hook_config` updates, the `agent` one without skill bodies (`skills: []`). - `F09.load-plugin-hook-runs`: after a runtime load, the next user turn runs the plugin's `UserPromptSubmit` hook (a `HookExecutionEvent` with its output) and finishes. - `F09.load-plugin-errors`: no registered marketplaces and no agent context are 400, a registered marketplace whose source is missing is 400, an unknown marketplace or plugin is 404, empty and malformed refs are 400, a missing `plugin_ref` is 422 and an unknown conversation 404; a bare plugin name resolves across the registered marketplaces. - `F09.load-plugin-twice`: loading the same plugin twice leaves one copy of its hooks, so they still run once per message. @@ -96,7 +96,10 @@ Routes: `POST /api/conversations/{conversation_id}/switch_profile`, `switchAcpEffort(id, effort)`; `RemoteConversation.switchProfile`, `switchLlm`, `switchAcpModel`, `switchAcpEffort`; - `ConversationManager.switchProfile`. There is no TypeScript `loadPlugin`. + `ConversationManager.switchProfile`; + `ConversationClient.loadPlugin(id, ref)` and + `PluginsClient.loadPluginForConversation(id, ref)` post + `{"plugin_ref": ref}` to `load_plugin` (driven by `F34.load-plugin`). `F09.ts-client` drives the `ConversationClient` methods and `ConversationManager.switchProfile` (the `RemoteConversation` methods post the same bodies to the same routes). @@ -805,14 +808,17 @@ Preconditions: --save F09.load-plugin/get control-agent-server state cat "server/workspace/conversations/${PID//-/}/base_state.json" --max-chars 200 \ --contains qa-f09-mkt-plugin-skill --check hook_config.user_prompt_submit len-eq 1 - control-agent-server ws read qa-f09-plugin --kinds ConversationStateUpdateEvent --contains qa-f09-mkt-plugin-skill --expect-min 1 --wait 20 --save F09.load-plugin/agent-frame + control-agent-server ws read qa-f09-plugin --kinds ConversationStateUpdateEvent --contains '"key": "agent"' --expect-min 1 --wait 20 --save F09.load-plugin/agent-frame + control-agent-server ws read qa-f09-plugin --contains qa-f09-mkt-plugin-skill --expect-none control-agent-server ws stop qa-f09-plugin --kinds ConversationStateUpdateEvent --contains QA_PLUGIN_HOOK --expect-min 1 --wait 20 ``` Before the load the agent has no skills and no hooks. After it, `GET` (with `include_skills=true`) lists the plugin's skill and its command (as the skill `qa-f09-mkt-plugin:qa-hello`), `hook_config` has the plugin's `UserPromptSubmit` hook, `base_state.json` has both, and the socket pushed - state updates for `agent` and `hook_config`. + state updates for `agent` and `hook_config`. The `agent` update (like the + `full_state` snapshot) carries `agent_context.skills` as `[]`, so no frame + names the plugin's skill; read skills from `GET` with `include_skills=true`. - **The loaded hook runs (`F09.load-plugin-hook-runs`).** Run one turn on `PID`. @@ -917,7 +923,7 @@ Preconditions: - **Restart (`F09.restart-persist`).** Restart the server and read every switched conversation back; then run a turn on `RID`. - Requires: `F09.switch-profile`, `F09.acp-model-deferred`, `F09.load-plugin` + Requires: `F09.switch-llm-ws`, `F09.switch-profile`, `F09.acp-model-deferred`, `F09.load-plugin` ```sh control-agent-server restart control-agent-server api GET "/api/conversations/$CID" --quiet --check agent.llm.model eq openai/qa-f09-ws --check agent.llm.usage_id eq qa-f09-ws --save F09.restart-persist/cid @@ -982,7 +988,9 @@ Preconditions: that the load is what adds the plugin. - Plugin commands become skills named `:`, and `GET /api/conversations/{id}` trims `agent.agent_context.skills` to `[]` - unless `include_skills=true`. + unless `include_skills=true`. Socket and webhook + `ConversationStateUpdateEvent`s (`full_state` and `agent`) always carry + `agent_context.skills` as `[]`; the persisted state keeps the skills. - `switch_acp_model` before the first run only persists the model; the provider's capability (`supports_runtime_model_switch`) and `available_models` are unknown (`false`, `[]`) until a session started. A diff --git a/.agents/skills/verify-agent-server/references/feature-map/F14-files.md b/.agents/skills/verify-agent-server/references/feature-map/F14-files.md index 699a02570ef..8981b8f3171 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F14-files.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F14-files.md @@ -80,17 +80,19 @@ Routes: `GET /api/file/home`, `GET /api/file/search_subdirs`, `F14.sdk-workspace` (an `exec` program, both scopes) and `F14.upload-size` (the upload result's `file_size`, against `LocalWorkspace.file_upload`). - TypeScript client: `FileClient.uploadFile`, `uploadTextFile`, `downloadFile`, - `downloadTextFile`, `downloadTrajectory` (scoped through the runtime - transport when a `conversationId` is set), `getHome`, - `searchSubdirectories`; `RemoteConversation.downloadTrajectory()` uses the + `downloadTextFile`, `downloadTrajectory`, `downloadArchive` (a `Blob`) + and `downloadArchiveBuffer` (an `ArrayBuffer`) with `{format, baseRef, + useDefaultExcludes, exclude}` (all scoped through the runtime transport + when a `conversationId` is set; the archive methods are driven by + `F34.file-archive`), `getHome`, `searchSubdirectories`; `RemoteConversation.downloadTrajectory()` uses the global trajectory route; `RemoteWorkspace.fileUpload`/`fileDownload` call `/api/file/upload|download` (the TypeScript `write_file` tool reports `fileUpload`'s `file_size`). `F14.upload-size` drives `RemoteWorkspace.fileUpload` from the built `clients/typescript/dist`. - Agent Canvas (context only): the workspace picker uses `home` and `search_subdirs`; file upload and "download trajectory" use the transfer - routes. `archive` and `create_directory` have no in-repo consumer; the - hosting app calls them to persist a workspace before deleting a runtime. + routes. `create_directory` has no in-repo consumer and `archive` none + but the TypeScript methods above; the hosting app calls them to persist a workspace before deleting a runtime. ## Driving it with control-agent-server diff --git a/.agents/skills/verify-agent-server/references/feature-map/F15-git.md b/.agents/skills/verify-agent-server/references/feature-map/F15-git.md index 72025c4452a..22f0b33634d 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F15-git.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F15-git.md @@ -108,7 +108,11 @@ Routes: `GET /api/conversations/{runtime_conversation_id}/git/changes`, `gitDiff(path, {ref})` (scoped through the runtime transport when a `conversationId` is set); `GitClient.searchRepositories({provider, query, limit, pageId})`. Driven by `F15.ts-client` (a `node` program on - `clients/typescript/dist`). + `clients/typescript/dist`). `GitClient.getCommits(path, {limit})` and + `getCommitChanges(path, sha)` (`GET /api/git/commits` and + `.../commits/{sha}/changes`, through the runtime transport, so scoped + when a `conversationId` is set), also as `ConversationManager.git` and + `AgentServerClient.git`; driven by `F34.git-commits`. - Agent Canvas (context only): the Changes tab (changes and diff), the Commits tab (commits, commit changes, diff with `commit`), the repository picker (search) and pull-request status badges. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F18-settings-and-secrets.md b/.agents/skills/verify-agent-server/references/feature-map/F18-settings-and-secrets.md index ac3d066fdbc..64e8a700780 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F18-settings-and-secrets.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F18-settings-and-secrets.md @@ -53,7 +53,7 @@ Routes: `GET /api/settings/agent-schema`, `GET /api/settings/conversation-schema - `F18.upgrade-from-release`: settings, an LLM key and a secret saved by the latest released server read back identical after the same state is restarted on this checkout (`restart --checkout`), the upgrade does not rewrite either file, and the experimental routes answer. - `F18.corrupted-files`: with an unparsable `settings.json`, GET silently returns defaults (every experimental feature off) and PATCH, of the settings or of an experimental feature, is 409 without touching the file; with an unparsable `secrets.json` the list is empty, GET by name 404, PUT and DELETE 500; restoring the files restores the values. - `F18.no-cipher`: without any cipher key, `X-Expose-Secrets: encrypted` is 503, `plaintext` still works, both files store plaintext, and the secrets routes are open without a key. -- `F18.secret-empty-value`: a PUT whose value is empty, blank or the redaction placeholder is refused with 422 and neither wipes an existing value nor leaves a listed but unreadable secret (fixed by #5682). +- `F18.secret-empty-value`: a PUT whose value is empty, blank or the redaction placeholder is refused with a validation 422 on `value` and neither wipes an existing value nor leaves a listed but unreadable secret. - `F18.dangling-pointers`: PATCH refuses an `active_agent_profile_id` or `active_meta_profile` that names nothing with 404, as the dedicated activate routes do, and stores nothing (fixed by #5847). - `F18.secret-scope-profile-errors`: `GET /api/settings/secrets?agent_profile_id=` naming an agent profile whose file no longer validates, or arriving while the agent-profile store's lock is held elsewhere, must answer as the agent-profile routes do (400 and 503), not with an unhandled 500. - `F18.secret-name-contract`: the published OpenAPI description of the secret routes names the status an invalid secret name actually gets, 422 (fixed by #5846). @@ -804,15 +804,17 @@ Preconditions: trap 'restore_token || true' EXIT control-agent-server api GET /api/settings/secrets/QA_F18_TOKEN --expect 200 --check . eq "$QA_F18_SECRET" control-agent-server api PUT /api/settings/secrets --json '{"name": "QA_F18_TOKEN", "value": "**********", "description": "qa f18 edited"}' \ - --expect 422 --check detail eq 'Secret value must not be empty or redacted' --save F18.secret-empty-value/put-placeholder + --expect 422 --check detail len-eq 1 --check detail.0.loc eq '["body", "value"]' --check detail.0.type eq value_error \ + --check detail.0.msg eq 'Value error, The secret must be a real value, not blank or redacted' --save F18.secret-empty-value/put-placeholder control-agent-server api GET /api/settings/secrets/QA_F18_TOKEN --expect 200 --check . eq "$QA_F18_SECRET" --save F18.secret-empty-value/get-after-placeholder control-agent-server api GET /api/settings/secrets --check secrets contains QA_F18_TOKEN --check . not-contains 'qa f18 edited' ``` The value reads back before the PUT (the control). The placeholder PUT is - refused with 422 `Secret value must not be empty or redacted` (#5682), and - the stored value and description are untouched. Before #5682 it answered - 200 and silently wiped the value (`CustomSecret`'s validator turns - `**********` and `""` into `null`), leaving a listed secret whose read was + refused with a standard validation 422: `detail` is a one-entry list with + `loc` `["body", "value"]`, `type` `value_error` and `msg` + `Value error, The secret must be a real value, not blank or redacted`; the stored value and description are untouched. + Accepting it would wipe the value (`CustomSecret`'s validator turns + `**********` and `""` into `null`), leaving a listed secret whose read is 404. The trap puts the secret back in case the PUT ever succeeds. - **Empty secret value (`F18.secret-empty-value`).** PUT a new secret whose value is the empty string, then one that is only whitespace. @@ -822,15 +824,15 @@ Preconditions: control-agent-server api GET /api/settings/secrets --check secrets contains QA_F18_TOKEN --check secrets not-contains QA_F18_EMPTY control-agent-server api GET /api/settings/secrets/QA_F18_TOKEN --expect 200 --check . eq "$QA_F18_SECRET" control-agent-server api PUT /api/settings/secrets --json '{"name": "QA_F18_EMPTY", "value": ""}' \ - --expect 422 --check detail eq 'Secret value must not be empty or redacted' --save F18.secret-empty-value/put-empty - control-agent-server api PUT /api/settings/secrets --json '{"name": "QA_F18_EMPTY", "value": " "}' --expect 422 + --expect 422 --check detail.0.loc eq '["body", "value"]' --check detail.0.msg eq 'Value error, The secret must be a real value, not blank or redacted' --save F18.secret-empty-value/put-empty + control-agent-server api PUT /api/settings/secrets --json '{"name": "QA_F18_EMPTY", "value": " "}' --expect 422 --check detail.0.msg eq 'Value error, The secret must be a real value, not blank or redacted' control-agent-server api GET /api/settings/secrets --check secrets not-contains QA_F18_EMPTY --save F18.secret-empty-value/list-empty control-agent-server api GET /api/settings/secrets/QA_F18_EMPTY --expect 404 ``` A listed secret is readable (the control: `QA_F18_TOKEN`). An empty or blank value is refused with 422 and leaves nothing behind: the name is not - listed and a read is 404. Before #5682 the PUT answered 200 and listed an - unreadable `QA_F18_EMPTY`. The trap deletes it in case a PUT succeeds. + listed and a read is 404; the 422 body is the same validation entry as for + the placeholder. The trap deletes it in case a PUT succeeds. - **Dangling agent-profile pointer (`F18.dangling-pointers`).** The activate route, and PATCH itself for an LLM profile name, refuse targets that do not exist; PATCH an `active_agent_profile_id` that names nothing. @@ -1016,7 +1018,7 @@ Preconditions: later `--rotate-secret-key` into a no-op. - `**********` (and an empty string) posted as a secret means "no secret" in a settings PATCH, where it clears the key; a secrets PUT refuses it (and - a blank value) with 422 since #5682 (`F18.secret-empty-value`). + a blank value) with a validation 422 on `value` (`F18.secret-empty-value`). Round-trip with `X-Expose-Secrets: encrypted`, which the server decrypts. - PATCH responses are always redacted, and `conversation_settings` is always redacted, whatever `X-Expose-Secrets` says. Any authenticated caller may ask diff --git a/.agents/skills/verify-agent-server/references/feature-map/F19-mcp.md b/.agents/skills/verify-agent-server/references/feature-map/F19-mcp.md index 6ff756f51a6..d0293488db2 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F19-mcp.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F19-mcp.md @@ -12,7 +12,10 @@ before a connection. Conversations started from these settings connect to every enabled server and expose its tools to the agent; disabled servers stay stored but are never spawned. When a conversation refreshes an expired OAuth token, the rotated tokens are written back into the stored server, or, for a server -passed inline on the agent, posted to the app server's webhook. +passed inline on the agent, posted to the app server's webhook. A conversation does not wait +for slow servers: it starts after a 1-second budget, and servers still +starting join the agent when ready, before its first model call. A running +conversation can also be given more servers through its own MCP route. Source: `openhands-agent-server/openhands/agent_server/mcp_router.py`, `openhands-agent-server/openhands/agent_server/mcp_oauth_store.py`, `openhands-agent-server/openhands/agent_server/settings_router.py`, `openhands-agent-server/openhands/agent_server/_secrets_exposure.py`, `openhands-sdk/openhands/sdk/mcp/`, `openhands-sdk/openhands/sdk/settings/api_models.py`, `openhands-sdk/openhands/sdk/utils/redact.py`, `openhands-sdk/openhands/sdk/workspace/remote/base.py`, `clients/typescript/src/client/mcp-client.ts`, `clients/typescript/src/client/settings-client.ts` @@ -24,7 +27,8 @@ Routes: `POST /api/mcp/test`, `POST /api/mcp/oauth/start`, `GET /api/mcp/oauth/status/{job_id}`, `POST /api/mcp/oauth/callback/{job_id}`, `GET /api/mcp/oauth/redirect`, `POST /api/settings/mcp/{settings_key}`, `PATCH /api/settings/mcp/{settings_key}`, -`DELETE /api/settings/mcp/{settings_key}` +`DELETE /api/settings/mcp/{settings_key}`, +`POST /api/conversations/{conversation_id}/mcp` ## Sub-features @@ -62,6 +66,13 @@ Routes: `POST /api/mcp/test`, `POST /api/mcp/oauth/start`, - `F19.oauth-refresh-writeback`: when a stored OAuth server's access token has expired, the next conversation refreshes it, keeps the server's tools, and writes the rotated token and new expiry back into the settings (encrypted on disk) without posting them to the webhook. - `F19.conversation-secrets-at-rest`: a conversation started from the settings persists its agent's MCP credentials (a disabled server's `env`, OAuth tokens) encrypted, and no MCP credential this family used appears in plaintext under the conversations directory or in the server log. - `F19.restart`: stored MCP servers and their credentials survive a restart (plaintext GET and the SDK's `RemoteWorkspace.get_mcp_config()`), while in-memory OAuth jobs are gone (404). +- `F19.conversation-mcp-add`: `POST /api/conversations/{conversation_id}/mcp` on a conversation whose agent is initialized starts the server before it answers 200 `{"success": true}`, `GET /api/conversations/{conversation_id}` and `base_state.json` show it in `agent.mcp_config`, the model's next request offers its tools as `_`, a call returns the tool's result, deleting the conversation stops the child, and `/server_info` advertises `conversation_mcp_install_v1` and `conversation_configuration_additions_v1`. +- `F19.conversation-mcp-add-before-init`: on a conversation without a first message the route only records the server (no process, fast answer), with `env` values encrypted by the settings API decrypted for the child and stored encrypted in `base_state.json`; the first message connects it like any configured server. +- `F19.conversation-mcp-add-errors`: an unknown conversation is 404, a server name the agent already has and an SSE server are 400 with the reason, a server without `command`/`url`, an unknown field and a non-UUID id are 422, none of them changes the agent's `mcp_config`, and a disabled server is stored without being spawned. +- `F19.conversation-mcp-add-start-failure`: a server that cannot start (a command that exits at once) is refused with the route's 400 and is not added (known bug [#6926](https://github.com/OpenHands/software-agent-sdk/issues/6926), fix in [#6928](https://github.com/OpenHands/software-agent-sdk/pull/6928): 500 Internal Server Error with `exception: Connection closed`). +- `F19.conversation-mcp-add-restart`: a server added at runtime is still in the conversation's agent after a restart, and the tools the agent already called keep working under the same names (known bug [#6927](https://github.com/OpenHands/software-agent-sdk/issues/6927), fix in [#6929](https://github.com/OpenHands/software-agent-sdk/pull/6929): with one server the tools come back unprefixed, so `qa-f19-rs_qa_echo` is `Tool ... not found`). +- `F19.conversation-mcp-slow-start`: a conversation whose stdio server delays its handshake by 8 s is created within a few seconds, the server is spawned, and the agent's first model request waits for it and offers its tools; the agent's call returns its result. +- `F19.conversation-mcp-late-failure`: a server that fails after the startup budget costs only its own tools: the conversation starts at once, the first step waits for the failure, the other server's tools are offered, the run finishes and the log names the failed server. ## How to get to it (agent POV) @@ -101,6 +112,16 @@ Routes: `POST /api/mcp/test`, `POST /api/mcp/oauth/start`, `{"server_url", "oauth_state"}` with `X-Session-API-Key` to every configured webhook's `/mcp-oauth-state`, so the app server can persist them (driven with `launch --webhook-sink` and `sink read`). +- REST, one conversation: `POST /api/conversations/{conversation_id}/mcp` + with `{"mcp_servers": {"": }}` adds servers to that + conversation's agent only (settings untouched). When the agent is already + initialized the route connects them before answering (their tools are + named `_`); before the first message it only records them. + Encrypted credential values are decrypted with the server's key first. + Clients detect it through `GET /server_info` `capabilities` + (`conversation_mcp_install_v1`, `conversation_configuration_additions_v1`). + The server side is the SDK's `LocalConversation.add_mcp_servers()` (S09); + `RemoteConversation` and the TypeScript client have no method for it. - SDK: `RemoteWorkspace.get_mcp_config()` reads the stored servers with plaintext credentials (exercised in `F19.restart` through `exec`). - TypeScript client: `MCPClient.testServer`, `startOAuth`, `getOAuthStatus`, @@ -917,27 +938,35 @@ Preconditions: route stores the real token (plaintext GET is not ciphertext), redacted in responses and encrypted on disk. - **Disabled servers stay off (`F19.disabled-server`).** Store the CLI's - stdio fixture, disable `qa-env`, and look at a new conversation's tools - without running the model: the conversation starts with a queued prompt - (`--no-run`), because the agent, and with it every MCP connection, is - initialized on the first message, not at creation. + stdio fixture, disable `qa-env`, and look at the tools a new conversation + offers its model. The conversation runs the settings' agent with its LLM + pointed at an `llm-stub` that finishes at once (`--llm-json`), so the + stub's recorded request is the tool list the model saw. The first + `SystemPromptEvent` is not that list: it holds only the tools of servers + that were ready within the 1-second startup budget + (`F19.conversation-mcp-slow-start`), and the stdio fixture takes longer. Requires: `F19.test-encrypted-env`, `F19.oauth-stored-state` ```sh control-agent-server api POST /api/settings/mcp/qa-f19 --json "$MCP" --expect 201 control-agent-server api PATCH /api/settings/mcp/qa-env --json '{"enabled":false}' --expect 200 --check agent_settings.mcp_config.qa-env.enabled eq false - DCID=$(control-agent-server conversation start --tools none --no-autotitle --prompt 'Say hi' --no-run --print-id) - control-agent-server api GET "/api/conversations/$DCID" --check execution_status eq idle + D_STUB=$(control-agent-server fixture llm-stub --name qa-f19-disabled-stub --step 'tool:finish:{"message":"QA_F19_DISABLED_DONE"}' --print-path) + DCID=$(control-agent-server conversation start --tools none --no-autotitle --prompt 'Say hi' --print-id \ + --llm-json "$(jq -nc --arg b "$D_STUB/v1" '{model: "openai/qa-stub", api_key: "qa-stub-key", base_url: $b, num_retries: 0}')") + control-agent-server conversation wait "$DCID" --until finished --timeout 90 + control-agent-server sink read --name qa-f19-disabled-stub --contains '"name": "qa-f19_qa_echo"' --expect-min 1 --save F19.disabled-server/model-request + control-agent-server sink read --name qa-f19-disabled-stub --contains '"name": "qa-oauth_read_subject"' --expect-min 1 + control-agent-server sink read --name qa-f19-disabled-stub --contains qa_env --expect-max 0 control-agent-server api GET "/api/conversations/$DCID/events/search" --query limit=1 --max-chars 300 --save F19.disabled-server/system-prompt \ - --check items.0.kind eq SystemPromptEvent --check . contains qa_echo --check . contains read_subject --check . not-contains qa_env + --check items.0.kind eq SystemPromptEvent --check . not-contains qa_env pgrep -fx "$MCP_CMDLINE" if pgrep -f "^$PY $F19/qa_f19_tools.py"; then echo 'the disabled server was spawned' >&2; false; fi control-agent-server api DELETE "/api/conversations/$DCID" --expect 200 for i in $(seq 1 20); do pgrep -fx "$MCP_CMDLINE" >/dev/null || break; sleep 0.5; done if pgrep -fx "$MCP_CMDLINE"; then echo 'the stdio child outlived its conversation' >&2; false; fi ``` - The first event lists `qa-f19_qa_echo` and `qa-oauth_read_subject` (two - enabled servers, so names carry the server prefix) and nothing from + The model's request offers `qa-f19_qa_echo` and `qa-oauth_read_subject` + (two enabled servers, so names carry the server prefix) and nothing from `qa-env`; the enabled stdio server runs while the disabled one never starts, and deleting the conversation stops the child. - **The agent uses the MCP tool (`F19.conversation-tool`).** A DeepSeek turn, @@ -1043,6 +1072,162 @@ Preconditions: back unchanged (also through `RemoteWorkspace.get_mcp_config()`), the finished job id is 404, and the bullet removes the fixtures and the three servers. +- **Add a server to a running conversation (`F19.conversation-mcp-add`).** + A conversation whose agent is initialized (a queued first message), with + an `llm-stub` model that calls the added tool and finishes; the CLI's + stdio fixture is added under the name `qa-f19-rt`. + ```sh + control-agent-server api GET /server_info --save F19.conversation-mcp-add/server-info \ + --check capabilities contains conversation_mcp_install_v1 --check capabilities contains conversation_configuration_additions_v1 + RT_STUB=$(control-agent-server fixture llm-stub --name qa-f19-rt-stub --step 'tool:qa-f19-rt_qa_echo:{"text":"added"}' --step 'tool:finish:{"message":"QA_F19_RT_DONE"}' --print-path) + RT_AGENT=$(jq -nc --arg b "$RT_STUB/v1" '{llm: {model: "openai/qa-stub", api_key: "qa-stub-key", base_url: $b, num_retries: 0}, tools: []}') + RT_MCP=$(control-agent-server fixture mcp-server --name qa-f19-rt | jq -c .mcp_server) + RT_CMDLINE=$(echo "$RT_MCP" | jq -r '[.command] + .args | join(" ")') + ACID=$(control-agent-server conversation start --body-json "$(jq -nc --argjson a "$RT_AGENT" '{agent: $a}')" --no-autotitle --prompt 'Use the added tool.' --no-run --print-id) + control-agent-server api GET "/api/conversations/$ACID" --check agent.mcp_config.qa-f19-rt missing --quiet + if pgrep -fx "$RT_CMDLINE"; then echo 'spawned before the add' >&2; false; fi + control-agent-server api POST "/api/conversations/$ACID/mcp" --json "$(jq -nc --argjson s "$RT_MCP" '{mcp_servers: {"qa-f19-rt": $s}}')" \ + --expect 200 --check success eq true --save F19.conversation-mcp-add/add + pgrep -fx "$RT_CMDLINE" + control-agent-server api GET "/api/conversations/$ACID" --check agent.mcp_config.qa-f19-rt.command eq "$PY" --check agent.mcp_config.qa-f19-rt.enabled eq true --save F19.conversation-mcp-add/get + control-agent-server state cat base_state.json --conversation "$ACID" --max-chars 300 --check agent.mcp_config.qa-f19-rt.command eq "$PY" + control-agent-server api POST "/api/conversations/$ACID/run" --expect 200 --quiet + control-agent-server conversation wait "$ACID" --until finished --timeout 90 + control-agent-server sink read --name qa-f19-rt-stub --contains '"name": "qa-f19-rt_qa_echo"' --expect-min 2 --save F19.conversation-mcp-add/model-requests + control-agent-server conversation events "$ACID" --kinds ObservationEvent --contains QA_MCP_ECHO:added --expect-kind ObservationEvent --save F19.conversation-mcp-add/observation + ``` + The add answers only after the child runs; the agent, its persisted state + and the model's next request all have the server (`qa-f19-rt_qa_echo`), + and the stubbed call returns `QA_MCP_ECHO:added`. The conversation stays + for the error bullet, which deletes it. +- **Add before the first message (`F19.conversation-mcp-add-before-init`).** + A conversation created without a message has no initialized agent. The + added server is the F19 tools fixture with an `env` value encrypted by the + settings API (read back from a short-lived settings entry). + ```sh + control-agent-server api POST /api/settings/mcp/qa-f19-rt-enc --json '{"command":"/bin/true","env":{"QA_F19_RT":"qa-f19-rt-secret"}}' --expect 201 --quiet + RT_ENC=$(control-agent-server api GET /api/settings --header 'X-Expose-Secrets: encrypted' --field agent_settings.mcp_config.qa-f19-rt-enc.env.QA_F19_RT --unsafe-unredacted) + control-agent-server api DELETE /api/settings/mcp/qa-f19-rt-enc --expect 200 --quiet + test "${RT_ENC#gAAAA}" != "$RT_ENC" + BI_STUB=$(control-agent-server fixture llm-stub --name qa-f19-bi-stub --step 'tool:qa_env:{"name":"QA_F19_RT"}' --step 'tool:finish:{"message":"QA_F19_BI_DONE"}' --print-path) + BI_AGENT=$(jq -nc --arg b "$BI_STUB/v1" '{llm: {model: "openai/qa-stub", api_key: "qa-stub-key", base_url: $b, num_retries: 0}, tools: []}') + BCID=$(control-agent-server conversation start --body-json "$(jq -nc --argjson a "$BI_AGENT" '{agent: $a}')" --no-autotitle --print-id) + control-agent-server api POST "/api/conversations/$BCID/mcp" --expect 200 --expect-max-ms 1000 --save F19.conversation-mcp-add-before-init/add \ + --json "$(jq -nc --argjson s "$TOOLS_SRV" --arg enc "$RT_ENC" '{mcp_servers: {"qa-f19-bi": ($s + {env: {QA_F19_RT: $enc}})}}')" + if pgrep -f "^$PY $F19/qa_f19_tools.py"; then echo 'spawned before the first message' >&2; false; fi + control-agent-server state cat base_state.json --conversation "$BCID" --max-chars 300 --check agent.mcp_config.qa-f19-bi.env.QA_F19_RT matches '^gAAAA' + control-agent-server state grep qa-f19-rt-secret --glob 'server/workspace/conversations/**/*' --expect-none + control-agent-server conversation send "$BCID" --text 'Read the variable.' --wait --until finished --timeout 90 + control-agent-server sink read --name qa-f19-bi-stub --contains '"name": "qa_env"' --expect-min 2 + control-agent-server conversation events "$BCID" --kinds ObservationEvent --contains QA_ENV:qa-f19-rt-secret --expect-kind ObservationEvent --save F19.conversation-mcp-add-before-init/observation + control-agent-server api DELETE "/api/conversations/$BCID" --expect 200 --quiet + ``` + The add answers within a second and spawns nothing; the stored server + keeps the env value as ciphertext. The first message connects it (one + server, so the tool is the bare `qa_env`) and the child sees the + decrypted value. +- **Refused additions (`F19.conversation-mcp-add-errors`).** + + Requires: `F19.conversation-mcp-add` + ```sh + NOPE=$("$PY" -c 'import uuid; print(uuid.uuid4())') + control-agent-server api POST "/api/conversations/$NOPE/mcp" --json '{"mcp_servers":{"qa-f19-nope":{"command":"/bin/true"}}}' --expect 404 + control-agent-server api POST "/api/conversations/$ACID/mcp" --json '{"mcp_servers":{"qa-f19-rt":{"command":"/bin/true"}}}' \ + --expect 400 --check detail contains 'MCP server already exists' --save F19.conversation-mcp-add-errors/duplicate + control-agent-server api POST "/api/conversations/$ACID/mcp" --json '{"mcp_servers":{"qa-f19-sse":{"url":"http://127.0.0.1:9/sse","transport":"sse"}}}' \ + --expect 400 --check detail eq 'Runtime MCP additions do not support SSE transport: qa-f19-sse' + control-agent-server api POST "/api/conversations/$ACID/mcp" --json '{"mcp_servers":{"qa-f19-bad":{}}}' --expect 422 + control-agent-server api POST "/api/conversations/$ACID/mcp" --json '{"mcp_servers":{"qa-f19-bad":{"command":"/bin/true","bogus":1}}}' --expect 422 + control-agent-server api POST /api/conversations/qa-f19-not-a-uuid/mcp --json '{"mcp_servers":{}}' --expect 422 + OFF_MARK="$F19/qa-f19-off-started" + OFF_BODY=$(jq -nc --arg c "touch $OFF_MARK" '{mcp_servers: {"qa-f19-off": {command: "/bin/sh", args: ["-c", $c], enabled: false}}}') + control-agent-server api POST "/api/conversations/$ACID/mcp" --expect 200 --json "$OFF_BODY" + control-agent-server api GET "/api/conversations/$ACID" --save F19.conversation-mcp-add-errors/after \ + --check agent.mcp_config.qa-f19-rt.command eq "$PY" --check agent.mcp_config.qa-f19-off.enabled eq false \ + --check agent.mcp_config.qa-f19-sse missing --check agent.mcp_config.qa-f19-bad missing --check agent.mcp_config.qa-f19-nope missing + test ! -e "$OFF_MARK" + control-agent-server api DELETE "/api/conversations/$ACID" --expect 200 --quiet + for i in $(seq 1 20); do pgrep -fx "$RT_CMDLINE" >/dev/null || break; sleep 0.5; done + if pgrep -fx "$RT_CMDLINE"; then echo 'the added stdio child outlived its conversation' >&2; false; fi + ``` + Each refusal leaves the agent's `mcp_config` as it was; the disabled + server is stored with `enabled: false` and its command never runs; deleting + the conversation stops the child the add started. +- **A server that cannot start (`F19.conversation-mcp-add-start-failure`), known bug #6926.** + The command exits at once, so the MCP connection closes during the + handshake. The route's documented refusal is 400 (`Invalid MCP server + update`); today the exception escapes as a 500. + ```sh + SF_AGENT='{"llm": {"model": "openai/qa-offline", "api_key": "qa-offline-key", "base_url": "http://127.0.0.1:9/v1", "num_retries": 0}, "tools": []}' + FCID=$(control-agent-server conversation start --body-json "$(jq -nc --argjson a "$SF_AGENT" '{agent: $a}')" --no-autotitle --prompt 'Never run.' --no-run --print-id) + control-agent-server api POST "/api/conversations/$FCID/mcp" --json '{"mcp_servers":{"qa-f19-dies":{"command":"/bin/true"}}}' --expect 400 --save F19.conversation-mcp-add-start-failure/add # bug: status eq 500 && response.body.exception eq 'Connection closed' + control-agent-server api GET "/api/conversations/$FCID" --max-chars 300 --check agent.mcp_config.qa-f19-dies missing --save F19.conversation-mcp-add-start-failure/after + ``` + The server is not added either way (`agent.mcp_config` keeps no + `qa-f19-dies`); only the status is wrong. +- **Added servers after a restart (`F19.conversation-mcp-add-restart`), known bug #6927.** + The stub calls the added tool, finishes, and after a restart and a new + message calls the same tool again. + ```sh + RS_STUB=$(control-agent-server fixture llm-stub --name qa-f19-rs-stub --step 'tool:qa-f19-rs_qa_echo:{"text":"one"}' \ + --step 'tool:finish:{"message":"QA_F19_RS_ONE"}' --step 'tool:qa-f19-rs_qa_echo:{"text":"two"}' --step 'tool:finish:{"message":"QA_F19_RS_TWO"}' --print-path) + RS_AGENT=$(jq -nc --arg b "$RS_STUB/v1" '{llm: {model: "openai/qa-stub", api_key: "qa-stub-key", base_url: $b, num_retries: 0}, tools: []}') + RS_MCP=$(control-agent-server fixture mcp-server --name qa-f19-rs | jq -c .mcp_server) + RCID=$(control-agent-server conversation start --body-json "$(jq -nc --argjson a "$RS_AGENT" '{agent: $a}')" --no-autotitle --prompt 'First.' --no-run --print-id) + control-agent-server api POST "/api/conversations/$RCID/mcp" --json "$(jq -nc --argjson s "$RS_MCP" '{mcp_servers: {"qa-f19-rs": $s}}')" --expect 200 --quiet + control-agent-server api POST "/api/conversations/$RCID/run" --expect 200 --quiet + control-agent-server conversation wait "$RCID" --until finished --timeout 90 + control-agent-server conversation events "$RCID" --kinds ObservationEvent --contains QA_MCP_ECHO:one --expect-kind ObservationEvent + control-agent-server restart + control-agent-server api GET "/api/conversations/$RCID" --check agent.mcp_config.qa-f19-rs.command eq "$PY" --save F19.conversation-mcp-add-restart/after-restart + control-agent-server conversation send "$RCID" --text 'Again.' --wait --until finished --timeout 90 + control-agent-server conversation events "$RCID" --kinds ObservationEvent --contains QA_MCP_ECHO:two --expect-kind ObservationEvent --save F19.conversation-mcp-add-restart/second-call # bug: matching eq 0 && kinds.AgentErrorEvent ge 1 + ``` + The server survives the restart in the conversation's agent and is + reconnected on the next message, but as the agent's only server its tools + come back bare (`qa_echo`), so the second call by the name the agent was + given ends in an `AgentErrorEvent` `Tool 'qa-f19-rs_qa_echo' not found`. +- **A slow server does not hold up the start (`F19.conversation-mcp-slow-start`).** + The F19 tools fixture with a handshake delayed by 8 s, inline on the + conversation's agent; the stub reads one variable through it. + ```sh + SL_STUB=$(control-agent-server fixture llm-stub --name qa-f19-slow-stub --step 'tool:qa_env:{"name":"QA_F19_SLOW"}' --step 'tool:finish:{"message":"QA_F19_SLOW_DONE"}' --print-path) + SL_AGENT=$(jq -nc --arg b "$SL_STUB/v1" --argjson s "$TOOLS_SRV" '{llm: {model: "openai/qa-stub", api_key: "qa-stub-key", base_url: $b, num_retries: 0}, tools: [], mcp_config: {"qa-f19-slow": ($s + {args: ($s.args + ["8"]), env: {QA_F19_SLOW: "qa-f19-slow-ok"}})}}') + T0=$SECONDS + SLCID=$(control-agent-server conversation start --body-json "$(jq -nc --argjson a "$SL_AGENT" '{agent: $a}')" --no-autotitle --prompt 'Read the variable.' --print-id) + test $((SECONDS - T0)) -le 5 + pgrep -f "^$PY $F19/qa_f19_tools.py 8" + control-agent-server conversation wait "$SLCID" --until finished --timeout 120 + test $((SECONDS - T0)) -ge 8 + control-agent-server sink read --name qa-f19-slow-stub --contains '"name": "qa_env"' --expect-min 2 --save F19.conversation-mcp-slow-start/model-requests + control-agent-server conversation events "$SLCID" --kinds ObservationEvent --contains QA_ENV:qa-f19-slow-ok --expect-kind ObservationEvent --save F19.conversation-mcp-slow-start/observation + control-agent-server api DELETE "/api/conversations/$SLCID" --expect 200 --quiet + ``` + `POST /api/conversations` returns while the server is still sleeping, the + run's first model request comes only after the server is ready and + already offers `qa_env`, and the call returns the server's environment + value. +- **A late server that fails (`F19.conversation-mcp-late-failure`).** Two + servers inline: the CLI's stdio fixture and a command that sleeps 6 s and + exits before any handshake. + ```sh + LF_STUB=$(control-agent-server fixture llm-stub --name qa-f19-late-stub --step 'tool:finish:{"message":"QA_F19_LATE_DONE"}' --print-path) + LF_AGENT=$(jq -nc --arg b "$LF_STUB/v1" --argjson ok "$MCP" '{llm: {model: "openai/qa-stub", api_key: "qa-stub-key", base_url: $b, num_retries: 0}, tools: [], mcp_config: {"qa-f19-ok": $ok, "qa-f19-dies-late": {command: "/bin/sh", args: ["-c", "sleep 6; exit 3"]}}}') + T0=$SECONDS + LCID=$(control-agent-server conversation start --body-json "$(jq -nc --argjson a "$LF_AGENT" '{agent: $a}')" --no-autotitle --prompt 'Finish.' --print-id) + test $((SECONDS - T0)) -le 4 + control-agent-server conversation wait "$LCID" --until finished --timeout 90 + test $((SECONDS - T0)) -ge 6 + control-agent-server sink read --name qa-f19-late-stub --contains '"name": "qa-f19-ok_qa_echo"' --expect-min 1 --save F19.conversation-mcp-late-failure/model-request + control-agent-server conversation events "$LCID" --kinds ObservationEvent --contains QA_F19_LATE_DONE --expect-kind ObservationEvent + control-agent-server logs --grep 'qa-f19-dies-late;' --expect-min 1 + control-agent-server api DELETE "/api/conversations/$LCID" --expect 200 --quiet + ``` + The start returns before the failing server gives up, the first step + waits for it, the model is offered the working server's prefixed tool, + the run finishes, and the log carries `MCP server startup failed for + qa-f19-dies-late; continuing without its tools: Connection closed` + (wrapped over several lines, so the recipe greps `qa-f19-dies-late;`). ## Gotchas @@ -1063,8 +1248,25 @@ Preconditions: `pgrep -fx`, or anchor with `^`. - A conversation connects to its MCP servers lazily: `POST /api/conversations` without `initial_message` spawns nothing and records no events; the first - message (even queued with `run: false`) initializes the agent, spawns the - stdio children and writes the `SystemPromptEvent` with the MCP tools. + message (even queued with `run: false`) initializes the agent and spawns + the stdio children. +- Conversation start waits at most 1 second (`MCP_STARTUP_BUDGET_SECONDS`) + for MCP servers. The `SystemPromptEvent` written then lists only the tools + of servers ready by that time and is never updated; servers still starting + (a Python FastMCP stdio server usually takes 1-3 s, so the CLI fixtures + do) join the agent when ready, and the first model step waits up to 30 s + for them. To see what the model is offered, read an `llm-stub`'s recorded + request (`sink read --name --contains '"name": ""'`), not the + first event. Tools added through `POST /api/conversations/{id}/mcp` are not + in it either. +- Tools of a server added through the conversation route are always named + `_` while the agent is running, but a restart (or an add + before the first message) names them like configured servers: bare when it + is the agent's only server (`F19.conversation-mcp-add-restart`). +- The conversation MCP route runs the connection inside the request: an + initialized conversation answers only after the server has started (or + failed, up to 30 s). Explicit `transport: sse` is refused, but a `/sse` URL + without `transport` is not inferred as SSE here and is tried as HTTP. - With two or more enabled servers, conversation tool names carry the server key as a prefix (`qa-f19_qa_echo`); with one server they are bare (`qa_echo`). Prompts and `--contains` checks should not depend on which. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F23-llm-catalog-and-connections.md b/.agents/skills/verify-agent-server/references/feature-map/F23-llm-catalog-and-connections.md index 8e8b26cdb8b..abd45caa6e7 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F23-llm-catalog-and-connections.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F23-llm-catalog-and-connections.md @@ -40,7 +40,7 @@ Routes: `GET /api/llm/providers`, `GET /api/llm/models`, - `F23.conn-create`: `POST /api/llm/provider-connections` answers 201 with a server-minted 32-hex `id`, the fields, equal `created_at`/`updated_at` and `api_key_set: true`, never the key; `provider` defaults to `custom`; the list shows the same entries in creation order. - `F23.conn-key-encrypted`: the connection file `home/.openhands/provider-connections/provider_connections.json` is mode `600` with `schema_version` 1 and a Fernet `gAAAAA` token as `api_key`; the plaintext key appears in no response and in no file under the server's home. - `F23.no-cipher`: on a server with neither `OH_SECRET_KEY` nor a session key (no cipher), a connection's key is stored in plaintext in the `600` file, the list still never returns it, and activating a linked profile applies it. -- `F23.conn-create-validation`: a missing or empty `api_key` or `display_name`, an empty `provider`, a 129-character `display_name`, and any extra field (`extra_headers`, a client-chosen `id`) are 422 and create nothing. +- `F23.conn-create-validation`: a missing or empty `api_key` or `display_name`, an empty `provider`, a 129-character `display_name`, and any extra field (`extra_headers`, a client-chosen `id`) are 422 and create nothing; an empty `api_key` reports `value_error` `The secret must be a real value, not blank or redacted`. - `F23.conn-create-blank-key`: a whitespace-only or redacted-placeholder (`**********`) `api_key` is refused like an empty one instead of creating a keyless connection. - `F23.linked-run`: a profile saved with `provider_connection_id` stores no inline key; activating it writes the connection's key into `agent_settings.llm`, and a conversation started from those settings runs on the connection's DeepSeek key and finishes. - `F23.conn-update`: `PATCH` renames a connection and sets or clears (`null`) its `base_url`, keeping `id`, `created_at` and the key; the list shows the change, and the next activation of a linked profile applies the connection's `base_url` (and clears it). @@ -49,7 +49,7 @@ Routes: `GET /api/llm/providers`, `GET /api/llm/models`, - `F23.conn-rotate-key`: rotating `api_key` leaves already-activated settings on the old key; the next activation (`PATCH /api/settings` `active_profile` or `POST /api/profiles/{name}/activate`) applies the new key, which is what the provider then receives, and rotating back makes conversations finish again. - `F23.validate-linked-draft`: `POST /api/profiles/{name}/validate` of a draft that references a connection checks it with the connection's key, so a connection holding a working DeepSeek key validates `valid: true` like the same key inline. - `F23.persist-restart`: connections (ids, names, timestamps, `api_key_set`) survive a restart unchanged, and a linked profile still resolves the key afterwards. -- `F23.strict-decrypt`: after the server's secret key changes, list, create, update and delete answer 400 `api_key is encrypted but cannot be decrypted with the current cipher` and leave the file byte-identical; with the old key back everything works and the key is intact. +- `F23.strict-decrypt`: after the server's secret key changes, list, create, update and delete answer 400 with a validation report per stored key, `The secret is encrypted but no configured key can decrypt it`, naming `OH_SECRET_KEY` and `OH_PREVIOUS_SECRET_KEYS`, and leave the file byte-identical; with the old key back everything works and the key is intact. - `F23.corrupt-file`: a connection file that is not valid JSON, or has a newer `schema_version`, makes the routes answer 400 without rewriting the file; restoring it restores the connections. - `F23.store-busy`: while another process holds the store's lock, a create waits out the 30-second lock timeout and answers 503 `Profile store is busy. Please retry.` without creating anything. - `F23.store-busy-responsive`: while a connection request waits for the store lock, the server keeps answering other requests (`GET /alive` within 3 seconds). @@ -321,7 +321,8 @@ Preconditions: control-agent-server api POST /api/llm/provider-connections --json '{"api_key": "sk-qa-f23-bad"}' --expect 422 \ --check detail.0.type eq missing --check detail.0.loc contains display_name control-agent-server api POST /api/llm/provider-connections --json '{"display_name": "qa-f23-bad", "api_key": ""}' --expect 422 \ - --check detail.0.type eq too_short --check detail.0.loc contains api_key + --check detail.0.type eq value_error --check detail.0.loc contains api_key \ + --check detail.0.msg contains 'The secret must be a real value, not blank or redacted' --save F23.conn-create-validation/empty-key control-agent-server api POST /api/llm/provider-connections --json '{"display_name": "", "api_key": "sk-qa-f23-bad"}' --expect 422 \ --check detail.0.type eq string_too_short --check detail.0.loc contains display_name control-agent-server api POST /api/llm/provider-connections --json '{"display_name": "qa-f23-bad", "provider": "", "api_key": "sk-qa-f23-bad"}' --expect 422 \ @@ -536,19 +537,20 @@ Preconditions: - **Secret key changed (`F23.strict-decrypt`).** Restart with a new `OH_SECRET_KEY`, try every connection route, then restore the old key. - Requires: `F23.conn-create` + Requires: `F23.conn-create`, `F23.linked-run` ```sh SUM=$(sha256sum "$CF" | cut -d' ' -f1) control-agent-server restart --rotate-secret-key control-agent-server api GET /api/llm/provider-connections --expect 400 \ - --check detail contains 'api_key is encrypted but cannot be decrypted with the current cipher' \ - --check detail contains OH_SECRET_KEY --save F23.strict-decrypt/list + --check detail contains 'connections.0.api_key' --check detail contains 'connections.1.api_key' \ + --check detail contains 'The secret is encrypted but no configured key can decrypt it' \ + --check detail contains OH_SECRET_KEY --check detail contains OH_PREVIOUS_SECRET_KEYS --save F23.strict-decrypt/list control-agent-server api POST /api/llm/provider-connections --json '{"display_name": "qa-f23-rotated", "api_key": "sk-qa-f23-rotated"}' \ - --expect 400 --check detail contains 'cannot be decrypted with the current cipher' + --expect 400 --check detail contains 'no configured key can decrypt it' control-agent-server api PATCH "/api/llm/provider-connections/$CONN" --json '{"display_name": "qa-f23-rotated"}' \ - --expect 400 --check detail contains 'cannot be decrypted with the current cipher' + --expect 400 --check detail contains 'no configured key can decrypt it' control-agent-server api DELETE "/api/llm/provider-connections/$DEF" --expect 400 \ - --check detail contains 'cannot be decrypted with the current cipher' --save F23.strict-decrypt/delete + --check detail contains 'no configured key can decrypt it' --save F23.strict-decrypt/delete test "$(sha256sum "$CF" | cut -d' ' -f1)" = "$SUM" control-agent-server restart --restore-secret-key control-agent-server api GET /api/llm/provider-connections --expect 200 --check . len-eq 2 --check 0.display_name eq qa-f23-deepseek @@ -556,7 +558,9 @@ Preconditions: control-agent-server api GET /api/settings --header 'X-Expose-Secrets: plaintext' --max-chars 200 \ --check agent_settings.llm.api_key eq "$DEEPSEEK_API_KEY" --save F23.strict-decrypt/restored ``` - Under the new key all four answers are 400 naming `OH_SECRET_KEY`, and + Under the new key all four answers are 400 whose `detail` is the + validation report of the stored file (one entry per connection key, + naming `OH_SECRET_KEY` and `OH_PREVIOUS_SECRET_KEYS`), and the file's SHA-256 is unchanged (nothing was rewritten or dropped). With the old key back both connections are listed and the DeepSeek key still resolves. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F25-skills.md b/.agents/skills/verify-agent-server/references/feature-map/F25-skills.md index 6675e5bb7a5..3046b4e98ce 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F25-skills.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F25-skills.md @@ -11,8 +11,12 @@ or disable, refresh from the recorded source, and uninstall. Enabled installed skills join the user tier of the catalog and of conversations that load user skills, where a trigger word activates them. The marketplace catalog lists the installable entries of the public repository with a fresh `installed` flag. +An operator can switch the public repository off: `OH_PUBLIC_SKILLS_ENABLED=false` +keeps its skills out of the catalogs and conversations, and +`OH_PUBLIC_CATALOG_ENABLED=false` empties the marketplace catalog, which then +says `disabled: true`. -Source: `openhands-agent-server/openhands/agent_server/skills_router.py`, `openhands-agent-server/openhands/agent_server/skills_service.py`, `openhands-sdk/openhands/sdk/skills/`, `openhands-sdk/openhands/sdk/extensions/`, `openhands-sdk/openhands/sdk/marketplace/`, `openhands-sdk/openhands/sdk/git/cached_repo.py`, `openhands-sdk/openhands/sdk/context/agent_context.py`, `openhands-sdk/openhands/sdk/workspace/remote/base.py`, `clients/typescript/src/client/skills-client.ts` +Source: `openhands-agent-server/openhands/agent_server/skills_router.py`, `openhands-agent-server/openhands/agent_server/skills_service.py`, `openhands-agent-server/openhands/agent_server/catalog.py`, `openhands-sdk/openhands/sdk/skills/`, `openhands-sdk/openhands/sdk/extensions/`, `openhands-sdk/openhands/sdk/marketplace/`, `openhands-sdk/openhands/sdk/git/cached_repo.py`, `openhands-sdk/openhands/sdk/context/agent_context.py`, `openhands-sdk/openhands/sdk/workspace/remote/base.py`, `clients/typescript/src/client/skills-client.ts` Needs: `git` @@ -59,6 +63,10 @@ Routes: `POST /api/skills`, `GET /api/skills/catalog`, `POST /api/skills/sync`, - `F25.ts-client`: the TypeScript `SkillsClient` installs a skill from a git source, lists, reads, disables and re-enables it (`toggleSkill`, with `getSkills` following the flag), refreshes, syncs, reads the marketplace and uninstalls it; an unknown skill is a 404 `HttpError` and a client without a key gets 401. - `F25.marketplace-stale-cache`: after a sync brings a new entry into the public repository, the catalog lists it: a successful sync clears the catalog's 5-minute cache, which never keeps an empty result. - `F25.uninstall`: uninstall deletes the directory and the metadata entry, the single view and a second uninstall are 404, the catalog drops the skill and the marketplace entry reads `installed: false` again. +- `F25.public-catalog-disabled`: with `OH_PUBLIC_CATALOG_ENABLED=false` the marketplace catalog answers `disabled: true` and no entries although the public clone lists some (the default server answers `disabled: false` with them), while public skills still load and sync still succeeds. +- `F25.public-skills-disabled`: with `OH_PUBLIC_SKILLS_ENABLED=false` neither `POST /api/skills`, the profile catalog nor a conversation with `load_public_skills` gets public skills (nothing is cloned, and a present clone is not read), while user, project and registered-marketplace skills still load and the marketplace catalog stays enabled. +- `F25.public-extensions-disabled`: with both switches off, sync answers `status: error` with `Public extensions are disabled by instance policy` and leaves the clone untouched, the marketplace is `disabled: true` and empty, and public skills do not load. +- `F25.load-registered-marketplaces`: `load_registered_marketplaces: true` loads auto-load registered marketplaces with `load_public: false`, `false` leaves them out with `load_public: true`, and leaving it out follows `load_public`. - `F25.long-description`: a skill description over 1024 characters is cut to 1024 in the project catalog, the installed view and the profile catalog, with a notice that names `invoke_skill(name=...)` and not the `SKILL.md` path. ## How to get to it (agent POV) @@ -66,7 +74,8 @@ Routes: `POST /api/skills`, `GET /api/skills/catalog`, `POST /api/skills/sync`, - REST, merged catalog: `POST /api/skills` with `SkillsRequest` (`load_public`, `load_user`, `load_project`, `load_org` all default to `true`, plus `project_dir`, `marketplace_path`, `registered_marketplaces`, - `org_configs`, `sandbox_config`) returns `{"skills": [SkillInfo], "sources": + `org_configs`, `sandbox_config`, and `load_registered_marketplaces`, which + defaults to `null`, meaning "follow `load_public`") returns `{"skills": [SkillInfo], "sources": {...}}`; `POST /api/skills/sync` force-updates the public repository clone. Driven by the `F25.catalog-*` and `F25.sync*` bullets. - REST, profile launch catalog: `GET /api/skills/catalog` returns @@ -104,7 +113,13 @@ Routes: `POST /api/skills`, `GET /api/skills/catalog`, `POST /api/skills/sync`, in the config file) adds server-default registrations that request entries override by name (driven through the config file by `F25.catalog-server-marketplaces`); `EXTENSIONS_REF` pins the public - repository ref. + repository ref. `OH_PUBLIC_SKILLS_ENABLED` and `OH_PUBLIC_CATALOG_ENABLED` + (default `true`; `false` turns one off, an unrecognised value keeps it + on, `F26.marketplace-disabled`) are read at call time: the first stops public skills + from loading anywhere, the second empties `GET /api/skills/marketplace` + (and the plugins marketplace, F26) and sets its `disabled` field; with both + off sync refuses (`F25.public-*-disabled`, each on a second server launched + with `--env`). Skills a client sends explicitly are not filtered. - WebSocket: `/sockets/events/{conversation_id}` carries the user `MessageEvent` with `activated_skills` (owned by the events family; the activation bullet replays it with `resend_mode=all`). @@ -889,6 +904,124 @@ Preconditions: full description is shown. Call invoke_skill(name="qa-f25-long") to load the full skill.`, naming the skill and never its file (the agent's prompt omits skill locations on purpose). +- **Public catalog switched off (`F25.public-catalog-disabled`).** The + run's marketplace (the local stand-in) is the default; a second server + launched with `OH_PUBLIC_CATALOG_ENABLED=false` gets its own clone of the + stand-in and reads its marketplace, public skills and sync. + + Requires: `F25.sync` + ```sh + control-agent-server api GET /api/skills/marketplace --expect 200 --check disabled eq false --check skills len-ge 1 \ + --check skills.0.name eq qa-f25-pub --save F25.public-catalog-disabled/default + NOCAT=$(control-agent-server launch --new --name f25-nocat --env OH_PUBLIC_CATALOG_ENABLED=false --print-run) + trap 'control-agent-server stop --run "$NOCAT" >/dev/null' EXIT + git clone -q "$E" "$NOCAT/home/.openhands/cache/skills/public-skills" + grep -q '"name": "qa-f25-pub"' "$NOCAT/home/.openhands/cache/skills/public-skills/marketplaces/default.json" + control-agent-server api GET /api/skills/marketplace --run "$NOCAT" --expect 200 --check disabled eq true --check skills len-eq 0 \ + --save F25.public-catalog-disabled/disabled + control-agent-server api POST /api/skills --run "$NOCAT" --expect 200 --check skills contains '"name": "qa-f25-pub"' \ + --json '{"load_public": true, "load_user": false, "load_project": false, "load_org": false}' --save F25.public-catalog-disabled/skills + control-agent-server api POST /api/skills/sync --run "$NOCAT" --expect 200 --check status eq success + control-agent-server stop --run "$NOCAT" + ``` + The default server's catalog says `disabled: false` and lists the + stand-in's entries. With the switch off, the same clone yields + `{"skills": [], "disabled": true}`, so a client can tell "turned off" from + "nothing to install"; the public tier of `POST /api/skills` still loads + `qa-f25-pub` and sync still succeeds, because only the catalog is off. +- **Public skills switched off (`F25.public-skills-disabled`).** A second + server launched with `OH_PUBLIC_SKILLS_ENABLED=false` and a user skill of + its own: load every tier before and after it has a clone of the stand-in, + read the profile catalog, the marketplace and a registered marketplace, + and create a never-run conversation (placeholder model) that asks for + public and user skills, next to the same conversation on the run's server. + + Requires: `F25.sync` + ```sh + KB='{"agent_settings": {"agent_kind": "openhands", "tools": [], "llm": {"model": "openai/qa-placeholder", "api_key": "qa-placeholder", "base_url": "http://127.0.0.1:9/v1", "num_retries": 0}, "agent_context": {"load_user_skills": true, "load_public_skills": true}}}' + KMAIN=$(control-agent-server conversation start --no-autotitle --body-json "$KB" --print-id) + control-agent-server api GET "/api/conversations/$KMAIN" --query include_skills=true --expect 200 \ + --check agent.agent_context.skills contains '"name": "qa-f25-pub"' --check agent.agent_context.skills contains '"name": "qa-f25-user"' + NOPUB=$(control-agent-server launch --new --name f25-nopub --env OH_PUBLIC_SKILLS_ENABLED=false --print-run) + trap 'control-agent-server stop --run "$NOPUB" >/dev/null' EXIT + NOPUB_CLONE="$NOPUB/home/.openhands/cache/skills/public-skills" + mkdir -p "$NOPUB/home/.agents/skills/qa-f25-nopub-user" + printf -- '---\nname: qa-f25-nopub-user\ndescription: QA user skill\n---\nQA_NOPUB_USER\n' > "$NOPUB/home/.agents/skills/qa-f25-nopub-user/SKILL.md" + QN='{"load_public": true, "load_user": true, "load_project": true, "load_org": false, "project_dir": "'"$P"'"}' + control-agent-server api POST /api/skills --run "$NOPUB" --json "$QN" --expect 200 --check skills contains '"name": "qa-f25-nopub-user"' \ + --check skills contains '"name": "qa-f25-proj-skill"' --check skills not-contains qa-f25-pub --check sources.sdk_base eq 1 + test ! -e "$NOPUB_CLONE" + git clone -q "$E" "$NOPUB_CLONE" + control-agent-server api POST /api/skills --run "$NOPUB" --json "$QN" --expect 200 --check skills contains '"name": "qa-f25-nopub-user"' \ + --check skills contains '"name": "qa-f25-proj-skill"' --check skills not-contains qa-f25-pub --check sources.sdk_base eq 1 \ + --save F25.public-skills-disabled/skills + control-agent-server api GET /api/skills/catalog --run "$NOPUB" --expect 200 --check skills contains '"name": "qa-f25-nopub-user"' \ + --check skills not-contains '"origin": "public"' --save F25.public-skills-disabled/profile-catalog + control-agent-server api GET /api/skills/marketplace --run "$NOPUB" --expect 200 --check disabled eq false --check skills.0.name eq qa-f25-pub + control-agent-server api POST /api/skills --run "$NOPUB" --expect 200 --check sources.registered_marketplaces eq 3 \ + --check skills contains '"name": "qa-f25-mk-skill"' --check skills not-contains '"name": "qa-f25-pub"' \ + --json '{"load_public": true, "load_user": false, "load_project": false, "load_org": false, "registered_marketplaces": [{"name": "qa-f25-mk", "source": "'"$M"'", "auto_load": true}]}' + KNOPUB=$(control-agent-server conversation start --run "$NOPUB" --no-autotitle --body-json "$KB" --print-id) + control-agent-server api GET "/api/conversations/$KNOPUB" --run "$NOPUB" --query include_skills=true --expect 200 \ + --check agent.agent_context.skills contains '"name": "qa-f25-nopub-user"' --check agent.agent_context.skills not-contains qa-f25-pub \ + --save F25.public-skills-disabled/conversation + control-agent-server stop --run "$NOPUB" + ``` + On the run's server the conversation gets the public stand-in skill and + the user skill. With the switch off nothing is cloned (no + `public-skills` directory appears) and even a present clone is not read: + every view has the user and project skills (`sources.sdk_base` counts the + one user skill) and no `qa-f25-pub`, the profile catalog has no `public` + origin, and the conversation's agent holds the user skill only. The + marketplace catalog stays enabled (`disabled: false`, its entry listed), + and a registered marketplace still contributes its three skills. +- **Both switches off (`F25.public-extensions-disabled`).** A third server + with both variables `false` and a clone of the stand-in one commit behind + its origin: sync, then read the marketplace and the public tier. + + Requires: `F25.sync` + ```sh + NOEXT=$(control-agent-server launch --new --name f25-noext --env OH_PUBLIC_SKILLS_ENABLED=false --env OH_PUBLIC_CATALOG_ENABLED=false --print-run) + trap 'control-agent-server stop --run "$NOEXT" >/dev/null' EXIT + NOEXT_CLONE="$NOEXT/home/.openhands/cache/skills/public-skills" + git clone -q "$E" "$NOEXT_CLONE" + git -C "$NOEXT_CLONE" reset -q --hard HEAD~1 + BEHIND=$(git -C "$NOEXT_CLONE" rev-parse HEAD) + test "$BEHIND" != "$(git -C "$E" rev-parse HEAD)" + control-agent-server api POST /api/skills/sync --run "$NOEXT" --expect 200 --check status eq error \ + --check message eq 'Public extensions are disabled by instance policy' --save F25.public-extensions-disabled/sync + test "$(git -C "$NOEXT_CLONE" rev-parse HEAD)" = "$BEHIND" + control-agent-server api GET /api/skills/marketplace --run "$NOEXT" --expect 200 --check disabled eq true --check skills len-eq 0 + control-agent-server api POST /api/skills --run "$NOEXT" --expect 200 --check skills not-contains qa-f25-pub --check sources.sdk_base eq 0 \ + --json '{"load_public": true, "load_user": false, "load_project": false, "load_org": false}' + control-agent-server stop --run "$NOEXT" + ``` + The sync answers `{"status": "error", "message": "Public extensions are + disabled by instance policy"}` without fetching: the clone's HEAD stays one + commit behind its origin (`F25.sync` shows the same request fast-forwarding + it on the default server). The catalog is `disabled: true` and empty, and + no public skill loads. +- **Registered marketplaces on their own (`F25.load-registered-marketplaces`).** + On the run's server, register `qa-f25-mk` and set + `load_registered_marketplaces` against `load_public` both ways, then leave + it out. + + Requires: `F25.sync` + ```sh + RM='"load_user": false, "load_project": false, "load_org": false, "registered_marketplaces": [{"name": "qa-f25-mk", "source": "'"$M"'", "auto_load": true}]' + control-agent-server api POST /api/skills --json '{"load_public": false, "load_registered_marketplaces": true, '"$RM"'}' --expect 200 \ + --check sources.registered_marketplaces eq 3 --check sources.sdk_base eq 0 --check skills contains '"name": "qa-f25-mk-skill"' \ + --check skills not-contains '"name": "qa-f25-pub"' --save F25.load-registered-marketplaces/registered-only + control-agent-server api POST /api/skills --json '{"load_public": true, "load_registered_marketplaces": false, '"$RM"'}' --expect 200 \ + --check sources.registered_marketplaces eq 0 --check skills contains '"name": "qa-f25-pub"' --check skills not-contains qa-f25-mk \ + --save F25.load-registered-marketplaces/public-only + control-agent-server api POST /api/skills --json '{"load_public": true, "load_registered_marketplaces": null, '"$RM"'}' --expect 200 \ + --check sources.registered_marketplaces eq 3 --check skills contains '"name": "qa-f25-pub"' + control-agent-server api POST /api/skills --json '{"load_public": false, '"$RM"'}' --expect 200 --check sources.registered_marketplaces eq 0 + ``` + `true` loads the marketplace's three skills without the public tier, + `false` keeps the public tier without them, and `null` or no field + couples them to `load_public` as before. ## Gotchas @@ -911,7 +1044,9 @@ Preconditions: - `sources` counts are per tier before the name merge (`sdk_base` is public plus user plus enabled installed skills), so their sum can exceed the number of skills. -- Registered marketplaces contribute only when `load_public` is true; the +- Registered marketplaces follow `load_public` unless the request sets + `load_registered_marketplaces` (`F25.load-registered-marketplaces`), and + `OH_PUBLIC_SKILLS_ENABLED=false` does not turn them off; the server-default registrations (`OH_REGISTERED_MARKETPLACES`) merge with the request's, the request winning by name. - The public-skills cache lives 60 s (forever for a tag or SHA diff --git a/.agents/skills/verify-agent-server/references/feature-map/F26-plugins.md b/.agents/skills/verify-agent-server/references/feature-map/F26-plugins.md index 228155b51c4..b347aabb0dc 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F26-plugins.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F26-plugins.md @@ -6,11 +6,12 @@ plugins a conversation would load on its own (user directories, the project's `.agents/plugins` and `.openhands/plugins`, and enabled installed plugins), installs plugins from a local path or a git source into the server's store, enables, disables, refreshes and uninstalls them, and reads the plugins-only -catalog of the public extensions marketplace. Enabled installed plugins are +catalog of the public extensions marketplace (empty and marked `disabled` +when the operator sets `OH_PUBLIC_CATALOG_ENABLED=false`). Enabled installed plugins are merged into every new conversation on its first message: their hooks run and their commands activate. -Source: `openhands-agent-server/openhands/agent_server/plugins_router.py`, `openhands-agent-server/openhands/agent_server/plugins_service.py`, `openhands-sdk/openhands/sdk/plugin/`, `openhands-sdk/openhands/sdk/extensions/`, `openhands-sdk/openhands/sdk/marketplace/`, `clients/typescript/src/client/plugins-client.ts` +Source: `openhands-agent-server/openhands/agent_server/plugins_router.py`, `openhands-agent-server/openhands/agent_server/plugins_service.py`, `openhands-agent-server/openhands/agent_server/catalog.py`, `openhands-sdk/openhands/sdk/plugin/`, `openhands-sdk/openhands/sdk/extensions/`, `openhands-sdk/openhands/sdk/marketplace/`, `clients/typescript/src/client/plugins-client.ts` Needs: `llm`, `git`, `node`, `network` @@ -48,6 +49,7 @@ Routes: `POST /api/plugins`, `POST /api/plugins/fetch`, `POST /api/plugins/insta - `F26.marketplace-local`: against a local stand-in for the extensions repository, `GET /api/plugins/marketplace` lists only `./plugins/` entries (local contents for local entries, attach coordinates for git entries) with a fresh `installed` flag; a repository without a manifest gives an empty catalog that is not cached. - `F26.marketplace-cache`: a non-empty catalog is served from a per-process cache (a new upstream entry appears only after a restart; the 5-minute TTL itself is not waited out), while `installed` stays fresh. - `F26.ts-client`: every `PluginsClient` method of the TypeScript client works against the server, and errors surface as `HttpError` with the status and detail. +- `F26.marketplace-disabled`: with `OH_PUBLIC_CATALOG_ENABLED=false` the marketplace answers `disabled: true` and no plugins although the clone lists some (the default server answers `disabled: false` with them); installing from an explicit source still works, and an unrecognised value keeps the catalog on. - `F26.marketplace-public`: against the real `OpenHands/extensions` repository, the catalog lists exactly its `./plugins/` entries, with their contents. ## How to get to it (agent POV) @@ -76,7 +78,9 @@ Routes: `POST /api/plugins`, `POST /api/plugins/fetch`, `POST /api/plugins/insta extensions clone at `$OH_PERSISTENCE_DIR/cache/skills/public-skills` (shared with the skills catalog, F25) and returns the `./plugins/` entries with attach coordinates (`source`, `ref`, `repo_path`), `installed`, and - `path`/`skills`/`files` when the entry is a directory of the clone. + `path`/`skills`/`files` when the entry is a directory of the clone, plus + `disabled` (`false` unless `OH_PUBLIC_CATALOG_ENABLED` is off, in which case + `plugins` is empty and nothing is fetched; `F26.marketplace-disabled`). - Conversations: `LocalConversation._ensure_plugins_loaded` merges enabled installed plugins, user-directory plugins and the workspace's project plugins into the agent on the first message or run. Second views: @@ -90,13 +94,18 @@ Routes: `POST /api/plugins`, `POST /api/plugins/fetch`, `POST /api/plugins/insta `installPlugin`, `listInstalledPlugins`, `getInstalledPlugin`, `setPluginEnabled`, `uninstallPlugin`, `refreshPlugin` (also as `OpenHandsClient.plugins`); driven by `F26.ts-client`. + `PluginsClient.loadPluginForConversation(id, ref)` calls F09's + `load_plugin` route through a `ConversationClient` (driven by + `F34.load-plugin`). - Python SDK: no remote client for these routes; `openhands.sdk.plugin` (`install_plugin`, `list_installed_plugins`, `load_available_plugins`, ...) is the same code run in-process. - Configuration: `OH_PERSISTENCE_DIR` (store and caches) and `HOME` (`~/.agents/plugins`) are read at import, so set them before launch (the launcher makes both private to the run); `EXTENSIONS_REF` pins the public - repository ref. + repository ref. `OH_PUBLIC_CATALOG_ENABLED` (default on, read per request) + switches the marketplace off for both the plugins and the skills catalogs + (F25); the bullet launches a second server with `--env`. - Agent Canvas (context only): the plugins marketplace page installs and attaches plugins from the catalog. @@ -748,7 +757,11 @@ Preconditions: - **Marketplace against a local stand-in (`F26.marketplace-local`).** Read the catalog while the stand-in has no manifest, then commit a manifest with a local plugin, a standalone skill and a plugin in another repository, read - it again, and install the local entry from its coordinates. + it again, and install the local entry from its coordinates. The entry in + another repository reads `installed: true` because `F26.install-git` + installed a plugin of that name. + + Requires: `F26.install-git` ```sh control-agent-server api GET /api/plugins/marketplace --expect 200 --check plugins len-eq 0 --save F26.marketplace-local/no-manifest test -d "$PUB/.git" @@ -858,6 +871,39 @@ Preconditions: lists the stand-in's three plugins, uninstall answers its message, and the errors reject with an `HttpError` whose `status` (and `detail`) are the server's. The server-side reads confirm the plugin is gone. +- **Catalog switched off (`F26.marketplace-disabled`).** The run's catalog + is the default; a second server launched with + `OH_PUBLIC_CATALOG_ENABLED=false` gets its own clone of the stand-in, reads + the catalog, installs a plugin from that clone directly, and is restarted + with an unrecognised value. + + Requires: `F26.marketplace-local` + ```sh + control-agent-server api GET /api/plugins/marketplace --expect 200 --check disabled eq false --check plugins len-ge 1 \ + --check plugins.0.name eq qa-f26-pub --save F26.marketplace-disabled/default + NOCAT=$(control-agent-server launch --new --name f26-nocat --env OH_PUBLIC_CATALOG_ENABLED=false --print-run) + trap 'control-agent-server stop --run "$NOCAT" >/dev/null' EXIT + NOCAT_CLONE="$NOCAT/home/.openhands/cache/skills/public-skills" + git clone -q "$E" "$NOCAT_CLONE" + grep -q '"name": "qa-f26-pub"' "$NOCAT_CLONE/.plugin/marketplace.json" + control-agent-server api GET /api/plugins/marketplace --run "$NOCAT" --expect 200 --check disabled eq true --check plugins len-eq 0 \ + --save F26.marketplace-disabled/disabled + control-agent-server api POST /api/plugins/install --run "$NOCAT" --json '{"source": "'"$NOCAT_CLONE/plugins/qa-f26-pub"'"}' --expect 200 \ + --check name eq qa-f26-pub + control-agent-server api GET /api/plugins/installed/qa-f26-pub --run "$NOCAT" --expect 200 --check enabled eq true + control-agent-server api GET /api/plugins/marketplace --run "$NOCAT" --expect 200 --check disabled eq true --check plugins len-eq 0 + control-agent-server restart --run "$NOCAT" --reset-config --env OH_PUBLIC_CATALOG_ENABLED=maybe + control-agent-server api GET /api/plugins/marketplace --run "$NOCAT" --expect 200 --check disabled eq false \ + --check plugins.0.name eq qa-f26-pub --check plugins.0.installed eq true --save F26.marketplace-disabled/unrecognised + control-agent-server stop --run "$NOCAT" + ``` + The default catalog says `disabled: false` and lists the stand-in's + plugins. With the switch off the same clone yields `{"plugins": [], + "disabled": true}`, so a client can tell "turned off" from "nothing to + install"; installing from an explicit source is not affected. After a + restart with `OH_PUBLIC_CATALOG_ENABLED=maybe` the server ignores the value, + keeps the catalog on, and lists the plugin as + installed. - **Public marketplace (`F26.marketplace-public`).** Remove the stand-in clone, restart, and read the catalog from the real `OpenHands/extensions` repository (network). diff --git a/.agents/skills/verify-agent-server/references/feature-map/F34-typescript-client.md b/.agents/skills/verify-agent-server/references/feature-map/F34-typescript-client.md index 62871386e72..1150bf57276 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F34-typescript-client.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F34-typescript-client.md @@ -45,7 +45,7 @@ Routes: none - `F34.ws-callback-client`: `RemoteConversation.startWebSocketClient()` in Node delivers the conversation's events to the conversation callback without errors. - `F34.conversation-controls`: `setTitle` and `setConfirmationPolicy` change the server's title and policy. - `F34.conversation-confirm`: under `AlwaysConfirm`, `sendConfirmationResponse(false, reason)` records one `UserRejectObservation` with that reason and leaves the command unrun, `sendConfirmationResponse(true)` runs the pending command, and `updateSecrets` (a string and a function value) puts both values into that command's shell without any event containing them. -- `F34.conversation-branching`: `fork({title})` returns a `RemoteConversation` bound to the fork (its id, the fork's workspace and agent, the copied events), `navigateTo` refreshes the cached state to the new `leaf_event_id`, and `askAgent` answers without adding a message or action to the source: the one event it adds is a `stats` `ConversationStateUpdateEvent` that reports the call's usage under `ask-agent-llm`. +- `F34.conversation-branching`: `fork({title})` returns a `RemoteConversation` bound to the fork (its id, the fork's workspace and agent, the copied events), `navigateTo` refreshes the cached state to the new `leaf_event_id`, and `askAgent` answers without adding a message or action to the source: the one event it adds (just after the call resolves) is a `stats` `ConversationStateUpdateEvent` that reports the call's usage under `ask-agent-llm`. - `F34.manager`: `ConversationManager` searches with paging, counts by status, batch-gets (unknown ids are `null`), lists all, loads a conversation with its server working directory, creates and deletes one, and an unknown id rejects with 404. - `F34.manager-update-result`: `ConversationManager.updateConversation` stores the new title and resolves with the updated `ConversationInfo` it is typed to return. - `F34.manager-create-options`: `ConversationManager.createConversation(agent, {maxIterations, stuckDetection})` creates the conversation with those settings. @@ -53,6 +53,11 @@ Routes: none - `F34.manager-acp`: `ConversationManager.acp` creates a conversation for an inline ACP agent (with its `maxIterations`) and gets, batch-gets, lists and counts it with its ACP agent fields. - `F34.hooks-client`: `HooksClient.loadHooks` and `RemoteConversation.loadHooks` read a project's `.openhands/hooks.json` as a snake_case `HookConfig` the client's hook helpers understand, and `null` for a directory without hooks. - `F34.mcp-client`: `MCPClient.testServer` lists a stdio MCP server's tools and runs one tool call, reports a server that fails to start as `{ok: false}` with an `error_kind`, and an empty command rejects with 422. +- `F34.events-batch-post`: `ConversationClient.batchGetEvents` posts to `POST .../events/batch` and returns the events in input order with `null` for an unknown id, exactly what the canonical GET batch returns, and rejects an unknown conversation (404) and a missing key (401) with an `HttpError`. +- `F34.load-plugin`: `ConversationClient.loadPlugin` and `PluginsClient.loadPluginForConversation` load a plugin from a conversation's registered marketplace (its skills and hook show in `GET`), and an unknown plugin, an empty ref, an unknown conversation and a missing key reject with the server's 404, 400, 404 and 401. +- `F34.git-commits`: `GitClient.getCommits` (with `limit`) and `getCommitChanges` (full or abbreviated SHA), through `ConversationManager.git`, `AgentServerClient.git` and a standalone `GitClient`, return the server's commit history and per-commit changes; `limit` 0 is 422, an unknown SHA 400, no key 401. +- `F34.file-archive`: `FileClient.downloadArchiveBuffer` and `downloadArchive` return a `tar.gz` (`ArrayBuffer`) and a git-delta patch (`Blob`) of a repository, pass `useDefaultExcludes` and repeated `exclude` globs and `baseRef` through, and reject a relative path or option-shaped `baseRef` (400) and a missing key (401). +- `F34.init-client`: `InitClient` (standalone, `AgentServerClient.init`, `ConversationManager.init`) reads a dormant server's status without a key, refuses `initialize` without or with a wrong `initApiKey` (401), initializes it with the dormant secret (`ready`), and a second `initialize` is a 400. - `F34.restart-reload`: after a server restart a new `ConversationManager` loads the same conversation with its status, title, events, hook config and workspace files. - `F34.event-stream-reconnect`: a `ConversationEventStream` with `reconnect: {enabled: true}` and `queryParams: {resend_mode: 'all'}`, on the `ws` package's constructor, reconnects by itself after a server restart, replays the history again with the same query, and delivers a message posted after the restart. - `F34.event-stream-reconnect-native`: the same stream on Node's global `WebSocket` also reconnects after a server restart. @@ -69,7 +74,8 @@ Routes: none `assertAgentServerSupports`, the hook helpers) and the secondary entry `dist/clients.js` (`ServerClient`, `FileClient`, `ConversationEventStream`, `buildConversationEventStreamUrl`, `AgentServerClient` and every endpoint - client). Recipes import the built files by path from the repository root. + client, including `ConversationClient`, `GitClient`, `InitClient` and + `PluginsClient`, which the main entry does not export). Recipes import the built files by path from the repository root. - Server status: `ServerClient.getAlive|getHealth|getReady|getServerInfo|getRoot` (`GET /alive`, `/health`, `/ready`, `/server_info`, `/`). - Conversations: `RemoteConversation.start()` (`POST /api/conversations`, or @@ -94,7 +100,26 @@ Routes: none does not call them. - State and events: `RemoteState` (`GET /api/conversations/{id}`, cached for 2 seconds), `RemoteEventsList.search|count|getEventById|getEventsById|getEvents` - (`GET .../events/search`, `.../events/count`, `.../events/{event_id}`). + (`GET .../events/search`, `.../events/count`, `.../events/{event_id}`); + `ConversationClient.batchGetEvents(id, ids)` (`POST .../events/batch`, the + browser-safe form of the GET batch; `F34.events-batch-post`). +- Plugins in a conversation: `ConversationClient.loadPlugin(id, ref)` and + `PluginsClient.loadPluginForConversation(id, ref)` (`POST + .../load_plugin` `{"plugin_ref": ref}`; `F34.load-plugin`). +- Commits: `GitClient.getCommits(path, {limit})` (`GET /api/git/commits`) and + `getCommitChanges(path, sha)` (`GET /api/git/commits/{sha}/changes`), on + `ConversationManager.git` and `AgentServerClient.git`; with a + `conversationId` they go through the runtime transport like `FileClient` + (`F34.git-commits`). +- Archives: `FileClient.downloadArchive(path, {format, baseRef, + useDefaultExcludes, exclude})` (a `Blob`) and `downloadArchiveBuffer` (an + `ArrayBuffer`), `GET /api/file/archive`, `exclude` sent as repeated query + parameters (`F34.file-archive`). +- Deferred init: `InitClient.getStatus()` (`GET /api/init`) and + `initialize(request, {initApiKey})` (`POST /api/init`, the key sent as + `X-Init-API-Key`, never as the session key), also as + `AgentServerClient.init` and `ConversationManager.init` + (`F34.init-client`). - Workspace: `RemoteWorkspace.executeCommand` (`POST /api/bash/execute_bash_command`), `uploadText`/`fileUpload` (`POST /api/file/upload`), `downloadAsText`/`downloadAsBlob`/`fileDownload` (`GET @@ -1112,7 +1137,7 @@ Preconditions: ENVIRONMENT=$(control-agent-server api GET "/api/conversations/$CID/events/count" --query source=environment --field .) cat > "$F34/branching.mjs" <<'JS' import { writeFileSync } from 'node:fs'; - import { m, host, apiKey, F34, assert } from './lib.mjs'; + import { m, host, apiKey, F34, assert, waitFor } from './lib.mjs'; const [cid, total] = process.argv.slice(2); const conversation = new m.RemoteConversation(new m.Agent({ llm: { model: 'qa-f34/unused' } }), new m.RemoteWorkspace({ host, apiKey, workingDir: '/x' }), { conversationId: cid }); @@ -1134,6 +1159,7 @@ Preconditions: const STATE = 'ConversationStateUpdateEvent'; const stateBefore = await conversation.state.events.count({ kind: STATE }); assert.equal(typeof (await conversation.askAgent('In one short sentence: which file did you create?')), 'string'); + await waitFor(async () => (await conversation.state.events.count()) > Number(total), 10000, 100); assert.equal(await conversation.state.events.count(), Number(total) + 1); assert.equal(await conversation.state.events.count({ kind: STATE }), stateBefore + 1); const [newest] = (await conversation.state.events.search({ limit: 1, sort_order: 'TIMESTAMP_DESC' })).items; @@ -1157,7 +1183,9 @@ Preconditions: before, `navigateTo()` leaves the cached state on the new HEAD, which GET confirms (the source's HEAD did not move). `askAgent` resolves with a string and adds exactly one event to the source, which is - still `finished`: the newest event is a `stats` + still `finished`. The server appends that event from a background task, so + it can land a few milliseconds after `askAgent` resolves; the program waits + up to 10 s for the count to grow before it asserts. The newest event is a `stats` `ConversationStateUpdateEvent` whose `usage_to_metrics` has an `ask-agent-llm` entry with the call's tokens, and the counts of every other kind are unchanged (the client's counts and REST agree: one more @@ -1389,6 +1417,201 @@ Preconditions: Expected: both streams come back, end at `{"isConnected": true, "isReconnecting": false, "attemptCount": 0}` and see the message posted after the restart. +- **Batch events by POST (`F34.events-batch-post`).** `F06.batch-post` + reached through the client, on `SCID`'s two events, then compared with + the canonical GET batch over REST. + ```sh + cat > "$F34/typed-batch.mjs" <<'JS' + import { writeFileSync } from 'node:fs'; + import { m, c, host, apiKey, F34, NOPE, assert } from './lib.mjs'; + const scid = process.argv[2]; + const conversations = new c.ConversationClient({ host, apiKey }); + const { items } = await conversations.searchEvents(scid, { limit: 2 }); + assert.equal(items.length, 2); + const [e0, e1] = items.map((e) => e.id); + const batch = await conversations.batchGetEvents(scid, [e1, NOPE, e0]); + assert.deepEqual(batch.map((e) => e && e.id), [e1, null, e0]); + assert.deepEqual(await new c.AgentServerClient({ host, apiKey }).conversations.batchGetEvents(scid, []), []); + await assert.rejects(conversations.batchGetEvents(NOPE, [e0]), (e) => m.isHttpError(e) && e.status === 404); + await assert.rejects(new c.ConversationClient({ host }).batchGetEvents(scid, [e0]), (e) => m.isHttpError(e) && e.status === 401); + writeFileSync(`${F34}/typed-batch.json`, JSON.stringify(batch)); + writeFileSync(`${F34}/typed-batch-ids.json`, JSON.stringify([e1, NOPE, e0])); + console.log('QA_F34_TYPED_BATCH_OK'); + JS + control-agent-server exec --timeout 60 --expect-output QA_F34_TYPED_BATCH_OK --save F34.events-batch-post/program -- node "$F34/typed-batch.mjs" "$SCID" + control-agent-server api GET "/api/conversations/$SCID/events" --json "$(cat "$F34/typed-batch-ids.json")" --expect 200 --quiet \ + --check . len-eq 3 --check 1 missing --raw-out "$F34/rest-batch.json" --save F34.events-batch-post/rest-get + diff <(jq -S . "$F34/rest-batch.json") <(jq -S . "$F34/typed-batch.json") + ``` + The program prints `QA_F34_TYPED_BATCH_OK`: `batchGetEvents` returns the + events in input order with `null` for the unknown id, `[]` for `[]` + (also through `AgentServerClient.conversations`), and rejects an unknown + conversation with a 404 and a missing key with a 401 `HttpError`. What + it returned equals the canonical GET batch for the same ids. +- **Load a plugin (`F34.load-plugin`).** `F09.load-plugin` reached through + the client: `PID` registers the `qa-f34-mkt` marketplace fixture on its + `agent_context` without loading it. + ```sh + MKT=$(control-agent-server fixture marketplace --name qa-f34-mkt --print-path) + AS=$(control-agent-server api GET /api/settings --header 'X-Expose-Secrets: encrypted' --field agent_settings --unsafe-unredacted \ + | jq -c --arg m "$MKT" '.tools = [] | .agent_context.registered_marketplaces = [{"name": "qa-f34-mkt", "source": $m}]') + PID=$(control-agent-server conversation start --no-autotitle --body-json "{\"agent_settings\": $AS, \"secrets_encrypted\": true}" --print-id) + control-agent-server api GET "/api/conversations/$PID" --query include_skills=true --quiet \ + --check agent.agent_context.skills len-eq 0 --check hook_config missing + cat > "$F34/typed-plugin.mjs" <<'JS' + import { m, c, host, apiKey, NOPE, assert } from './lib.mjs'; + const pid = process.argv[2]; + const conversations = new c.ConversationClient({ host, apiKey }); + await assert.rejects(conversations.loadPlugin(pid, 'qa-f34-nope@qa-f34-mkt'), (e) => m.isHttpError(e) && e.status === 404 && + e.detail === "Plugin 'qa-f34-nope' not found in marketplace 'qa-f34-mkt'"); + await assert.rejects(conversations.loadPlugin(pid, ''), (e) => m.isHttpError(e) && e.status === 400); + await assert.rejects(conversations.loadPlugin(NOPE, 'qa-f34-mkt-plugin@qa-f34-mkt'), (e) => m.isHttpError(e) && e.status === 404); + await assert.rejects(new c.ConversationClient({ host }).loadPlugin(pid, 'qa-f34-mkt-plugin@qa-f34-mkt'), + (e) => m.isHttpError(e) && e.status === 401); + const plugins = new c.AgentServerClient({ host, apiKey }).plugins; + assert.deepEqual(await plugins.loadPluginForConversation(pid, 'qa-f34-mkt-plugin@qa-f34-mkt'), { success: true }); + assert.deepEqual(await conversations.loadPlugin(pid, 'qa-f34-mkt-plugin@qa-f34-mkt'), { success: true }); + console.log('QA_F34_TYPED_PLUGIN_OK'); + JS + control-agent-server exec --timeout 60 --expect-output QA_F34_TYPED_PLUGIN_OK --save F34.load-plugin/program -- node "$F34/typed-plugin.mjs" "$PID" + control-agent-server api GET "/api/conversations/$PID" --query include_skills=true --quiet --check agent.agent_context.skills len-eq 2 \ + --check agent.agent_context.skills contains qa-f34-mkt-plugin-skill --check agent.agent_context.skills contains 'qa-f34-mkt-plugin:qa-hello' \ + --check hook_config.user_prompt_submit len-eq 1 --save F34.load-plugin/get + ``` + The program prints `QA_F34_TYPED_PLUGIN_OK`: an unknown plugin is a 404 + `HttpError` with the server's `detail`, an empty ref 400, an unknown + conversation 404, no key 401; `PluginsClient.loadPluginForConversation` + and then `ConversationClient.loadPlugin` (a second load) both resolve to + `{success: true}`. `GET` shows the plugin's skill and command skill once + each and its one `UserPromptSubmit` hook. +- **Commits (`F34.git-commits`).** `F15.commits` and `F15.commit-changes` + reached through `GitClient` (as `ConversationManager.git`, + `AgentServerClient.git` and standalone), on a fresh `git-repo` fixture + (commits `Initial commit` and `Add util`). + ```sh + TREPO=$(control-agent-server fixture git-repo --name qa-f34-typed-repo --print-path) + THEAD=$(git -C "$TREPO" rev-parse HEAD) + TROOT=$(git -C "$TREPO" rev-parse HEAD~1) + cat > "$F34/typed-git.mjs" <<'JS' + import { writeFileSync } from 'node:fs'; + import { m, c, host, apiKey, F34, assert } from './lib.mjs'; + const [repo, head, root] = process.argv.slice(2); + const manager = new m.ConversationManager({ host, apiKey }); + const commits = await manager.git.getCommits(repo); + assert.deepEqual(commits.commits.map((x) => [x.sha, x.subject]), [[head, 'Add util'], [root, 'Initial commit']]); + assert.equal(commits.has_more, false); + const first = await new c.AgentServerClient({ host, apiKey }).git.getCommits(repo, { limit: 1 }); + assert.deepEqual([first.commits.length, first.commits[0].sha, first.has_more], [1, head, true]); + const changes = await manager.git.getCommitChanges(repo, head.slice(0, 7)); + assert.deepEqual(changes, [{ status: 'ADDED', path: 'src/util.py' }]); + assert.deepEqual(await new c.GitClient({ host, apiKey }).getCommitChanges(repo, root), + [{ status: 'ADDED', path: 'README.md' }, { status: 'ADDED', path: 'src/app.py' }]); + await assert.rejects(manager.git.getCommits(repo, { limit: 0 }), (e) => m.isHttpError(e) && e.status === 422 && + e.validationErrors[0].loc.join('.') === 'query.limit'); + await assert.rejects(manager.git.getCommitChanges(repo, 'deadbeef'), (e) => m.isHttpError(e) && e.status === 400); + await assert.rejects(new c.GitClient({ host }).getCommits(repo), (e) => m.isHttpError(e) && e.status === 401); + writeFileSync(`${F34}/typed-commits.json`, JSON.stringify(commits)); + console.log('QA_F34_TYPED_GIT_OK'); + JS + control-agent-server exec --timeout 60 --expect-output QA_F34_TYPED_GIT_OK --save F34.git-commits/program -- node "$F34/typed-git.mjs" "$TREPO" "$THEAD" "$TROOT" + control-agent-server api GET /api/git/commits --query path="$TREPO" --expect 200 --quiet --raw-out "$F34/rest-commits.json" --save F34.git-commits/rest + diff <(jq -S . "$F34/rest-commits.json") <(jq -S . "$F34/typed-commits.json") + ``` + The program prints `QA_F34_TYPED_GIT_OK`: `getCommits` lists both commits + newest first with `has_more: false`, `{limit: 1}` cuts it to `HEAD` with + `has_more: true`, `getCommitChanges` takes an abbreviated SHA and lists + `ADDED src/util.py` (the root commit: both its files as `ADDED`); a + `limit` of 0 is a 422 on `query.limit`, an unknown SHA 400, no key 401. + The commits it returned equal REST's. +- **Archives (`F34.file-archive`).** `F14.archive-tar`, + `F14.archive-excludes` and `F14.archive-git-delta` reached through + `FileClient`, on the same repository (one modified, one untracked file). + + Requires: `F34.git-commits` + ```sh + cat > "$F34/typed-archive.mjs" <<'JS' + import { writeFileSync } from 'node:fs'; + import { m, c, host, apiKey, F34, assert } from './lib.mjs'; + const repo = process.argv[2]; + const files = new c.FileClient({ host, apiKey }); + const tar = await files.downloadArchiveBuffer(repo, { format: 'tar.gz' }); + assert.ok(tar instanceof ArrayBuffer); + writeFileSync(`${F34}/typed-archive.tgz`, Buffer.from(tar)); + const trimmed = await new c.AgentServerClient({ host, apiKey }).files.downloadArchiveBuffer(repo, + { format: 'tar.gz', useDefaultExcludes: false, exclude: ['notes.txt', 'src/*'] }); + writeFileSync(`${F34}/typed-archive-trimmed.tgz`, Buffer.from(trimmed)); + const delta = await files.downloadArchive(repo, { baseRef: 'HEAD' }); + assert.ok(delta instanceof Blob); + writeFileSync(`${F34}/typed-archive.patch`, Buffer.from(await delta.arrayBuffer())); + await assert.rejects(files.downloadArchive('qa-relative'), (e) => m.isHttpError(e) && e.status === 400); + await assert.rejects(files.downloadArchive(repo, { baseRef: '-qa' }), (e) => m.isHttpError(e) && e.status === 400); + await assert.rejects(new c.FileClient({ host }).downloadArchiveBuffer(repo), (e) => m.isHttpError(e) && e.status === 401); + console.log('QA_F34_TYPED_ARCHIVE_OK'); + JS + control-agent-server exec --timeout 60 --expect-output QA_F34_TYPED_ARCHIVE_OK --save F34.file-archive/program -- node "$F34/typed-archive.mjs" "$TREPO" + tar -tzf "$F34/typed-archive.tgz" > "$F34/typed-archive.list" + grep -qx 'qa-f34-typed-repo/notes.txt' "$F34/typed-archive.list" + grep -qx 'qa-f34-typed-repo/src/util.py' "$F34/typed-archive.list" + grep -qx 'qa-f34-typed-repo/archive_manifest.json' "$F34/typed-archive.list" + if grep -q '/\.git/' "$F34/typed-archive.list"; then false; fi + tar -tzf "$F34/typed-archive-trimmed.tgz" > "$F34/typed-archive-trimmed.list" + grep -qx 'qa-f34-typed-repo/.git/HEAD' "$F34/typed-archive-trimmed.list" + if grep -Eq 'notes\.txt|src/.+' "$F34/typed-archive-trimmed.list"; then false; fi + tar -xzOf "$F34/typed-archive-trimmed.tgz" qa-f34-typed-repo/archive_manifest.json > "$F34/typed-archive-manifest.json" + control-agent-server assert --json "$F34/typed-archive-manifest.json" --check excludes len-eq 2 --check excludes.0 eq notes.txt --check excludes.1 eq 'src/*' + grep -q '^+Modified, not committed\.' "$F34/typed-archive.patch" + grep -q '^+++ b/notes.txt' "$F34/typed-archive.patch" + ``` + The program prints `QA_F34_TYPED_ARCHIVE_OK`: `downloadArchiveBuffer` + with `format: 'tar.gz'` resolves to an `ArrayBuffer` holding the + repository with its manifest and without `.git/`; with + `useDefaultExcludes: false` and two `exclude` globs (sent as repeated + query parameters) `.git/` is in, `notes.txt` and `src/*` are out, and the + manifest records both globs; `downloadArchive` with `baseRef: 'HEAD'` + resolves to a `Blob` with the git-delta patch of the uncommitted and + untracked changes. A relative path and an option-shaped `baseRef` are + 400 `HttpError`s, no key 401. +- **Deferred init (`F34.init-client`).** `F03.dormant-boot`, + `F03.init-key-auth`, `F03.init-success` and `F03.init-second-call-400` + reached through `InitClient`, on a second, dormant server launched and + stopped inside the bullet. + ```sh + IRUN=$(control-agent-server launch --new --no-auth --deferred-init --name f34-init --print-run) + trap 'control-agent-server stop --run "$IRUN" > /dev/null' EXIT + ISK="$(cat "$IRUN/private/secret_key")" + test -n "$ISK" + control-agent-server api GET /api/init --run "$IRUN" --auth none --expect 200 --check state eq dormant + cat > "$F34/typed-init.mjs" <<'JS' + import { m, c, host, assert } from './lib.mjs'; + const initKey = process.argv[2]; + const init = new c.InitClient({ host }); + assert.deepEqual(await init.getStatus(), { state: 'dormant', error: null }); + await assert.rejects(init.initialize({ session_api_keys: ['qa-f34-init-key'] }), (e) => m.isHttpError(e) && e.status === 401); + await assert.rejects(init.initialize({ session_api_keys: ['qa-f34-init-key'] }, { initApiKey: 'qa-f34-wrong' }), + (e) => m.isHttpError(e) && e.status === 401); + assert.deepEqual(await new c.AgentServerClient({ host }).init.getStatus(), { state: 'dormant', error: null }); + assert.deepEqual(await init.initialize({ session_api_keys: ['qa-f34-init-key'] }, { initApiKey: initKey }), { state: 'ready', error: null }); + const manager = new m.ConversationManager({ host, apiKey: 'qa-f34-init-key' }); + assert.deepEqual(await manager.init.getStatus(), { state: 'ready', error: null }); + assert.equal(await manager.countConversations(), 0); + await assert.rejects(manager.init.initialize({}, { initApiKey: initKey }), (e) => m.isHttpError(e) && e.status === 400 && + e.detail.includes('already in state: ready')); + console.log('QA_F34_TYPED_INIT_OK'); + JS + control-agent-server exec --run "$IRUN" --timeout 60 --expect-output QA_F34_TYPED_INIT_OK --save F34.init-client/program -- node "$F34/typed-init.mjs" "$ISK" + control-agent-server api GET /api/init --run "$IRUN" --auth none --expect 200 --check state eq ready --save F34.init-client/ready + control-agent-server api GET /api/settings --run "$IRUN" --auth none --header 'X-Session-API-Key: qa-f34-init-key' --expect 200 + control-agent-server stop --run "$IRUN" + trap - EXIT + ``` + The program prints `QA_F34_TYPED_INIT_OK`: `getStatus` (also through + `AgentServerClient.init`) reads `{state: 'dormant', error: null}` without + a key; `initialize` without `initApiKey` or with a wrong one is a 401 + `HttpError`; with the dormant secret it sends `X-Init-API-Key` and + resolves to `{state: 'ready', error: null}`; a `ConversationManager` + with the delivered session key reads `ready` through `init` and counts + zero conversations, and a second `initialize` is a 400 `already in state: + ready`. REST agrees, and the delivered key opens `/api/settings`. - **Endpoint audit against this checkout (`F34.endpoint-audit`).** The client's own drift tooling, offline, in a fixture directory (nothing is written into `clients/typescript`). @@ -1405,6 +1628,8 @@ Preconditions: jq -e '.clientOnly == [] and .client > 100' "$A/.audit/endpoint-audit.json" jq -e '[.allowedClientOnly[].endpoint] == ["GET /", "GET /alive", "GET /health", "GET /ready", "GET /server_info"]' "$A/.audit/endpoint-audit.json" jq -r '.serverOnly[]' "$A/.audit/endpoint-audit.json" > "$A/server-only.txt" + if grep -E '^(GET|POST) /api/(init|file/archive|git/commits|conversations/\{\}/(events|load_plugin))' "$A/server-only.txt"; then false; fi + jq -e '[.explicitlyCoveredServerOperations[].endpoint] | index("GET /api/conversations/{}/events") != null' "$A/.audit/endpoint-audit.json" control-agent-server exec --env AGENT_SERVER_OPENAPI_PATH="$A/openapi.json" --env AGENT_SERVER_GENERATED_OUTPUT="$A/agent-server-schema.ts" \ --timeout 180 --save F34.endpoint-audit/generate -- node clients/typescript/scripts/generate-agent-server-api.mjs node clients/typescript/scripts/summarize-agent-server-api.mjs --before clients/typescript/src/generated/agent-server-schema.ts \ @@ -1420,13 +1645,18 @@ Preconditions: the recipe asserts `clientOnly == []` on the JSON itself. It reports no client-only call (every handwritten call exists on this checkout); `server-only.txt` lists the operations without a literal client - call (43 on the baseline: the conversation-scoped bash, file, git and VS + call (42 on this checkout: the conversation-scoped bash, file, git and VS Code aliases the client reaches by rewriting paths at runtime, the canvas extension routes, whose backend calls a helper builds, provider - connections, `/api/init`, the `GET .../events` batch, archives, commits, - `create_directory` and `load_plugin`). Regenerating the client's types - from this checkout removes no exported type (on the baseline the diff is - `+13 / -1` lines: two new fields of the canvas backend artifact). + connections, `create_directory`, pull-request status, the skills catalog, + `plugins/fetch`, conversation `children` and `mcp`). `/api/init`, the + global archive and commits routes and `load_plugin` have typed methods + now and are not in it; the `GET .../events` batch is listed under + `explicitlyCoveredServerOperations`, covered by `batchGetEvents` through + `POST .../events/batch`. Regenerating the client's types from this + checkout removes no exported type (the diff is large, `+1488 / -36` lines + on this checkout, because `src/generated/` follows the pinned release, + not this checkout). - **Old server version (`F34.version-gate-old`).** A real 1.22.0 release of the agent server, installed with `uvx`, attached and stopped inside the bullet. FastAPI is pinned to the version in the diff --git a/.agents/skills/verify-agent-server/references/feature-map/F38-conversation-mentions.md b/.agents/skills/verify-agent-server/references/feature-map/F38-conversation-mentions.md index 47573e599a9..7d7d7455225 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F38-conversation-mentions.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F38-conversation-mentions.md @@ -126,3 +126,4 @@ Preconditions: points at that variable, not at the flag. - The flag picks the tools when a conversation starts: switching it on does not give an open conversation the tools. Start a new one. +- Agent Canvas does not render mentions yet: its markdown sanitizer drops the `conversation:` scheme, so a mention shows as a link with an empty `href` (follow-up [OpenHands/OpenHands#19814](https://github.com/OpenHands/OpenHands/issues/19814)). diff --git a/.agents/skills/verify-agent-server/references/feature-map/F39-message-edits.md b/.agents/skills/verify-agent-server/references/feature-map/F39-message-edits.md index 460fd2b978e..cc7600049a4 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F39-message-edits.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F39-message-edits.md @@ -145,3 +145,4 @@ Preconditions: one response all apply. - Turning the flag off keeps `edit_message` in conversations started while it was on, as with reactions. +- Agent Canvas does not handle `MessageEditEvent` yet (follow-up [OpenHands/OpenHands#19793](https://github.com/OpenHands/OpenHands/issues/19793)). diff --git a/.agents/skills/verify-agent-server/references/feature-map/F40-ask-user.md b/.agents/skills/verify-agent-server/references/feature-map/F40-ask-user.md new file mode 100644 index 00000000000..ce19eaea350 --- /dev/null +++ b/.agents/skills/verify-agent-server/references/feature-map/F40-ask-user.md @@ -0,0 +1,358 @@ +# Decision cards (ask_user) + +When a choice is the user's to make, the agent can ask one question with two +to four options and a recommendation, which a client shows as a card the user +answers in one tap or in their own words. The question is the agent's +`ask_user` tool call: an `ActionEvent` (`tool_name: "ask_user"`) whose +`action` is an `AskUserAction` (`question`, `options` of `{label, +consequence}`, `recommended` index or null), followed by its +`AskUserObservation` saying the question was shown. The call ends the agent's +turn without `finish`, so the conversation is `finished` while the question +waits. A question is pending until a message of the user's own follows it. +The user answers either by writing in the chat (an ordinary message) or by +picking on the card through `POST /api/conversations/{id}/ask_user/answer`, +which saves an `AskUserAnswerEvent` (what the card shows as answered), then +the user message the agent reads (`[Answer to your question: "..."]`, +`Option N: label`, the user's words), and runs the agent. While a question +waits, nothing answers it for the user: a Stop hook that denies completion +does not run over it, and a `/goal` loop stops on it (`interrupted`) and +refuses to start or resume until the user answers. + +Decision cards sit behind the user-facing feature flag `ask_user` ("Decision +cards", off by default). While it is on, every new OpenHands-agent +conversation gets the `ask_user` tool without naming it, and the answer route +accepts answers; while it is off, the route answers 409 and new conversations +do not get the tool. Flags are listed with `GET /api/settings/experimental` +and switched with `PATCH /api/settings/experimental/{key}` +`{"enabled": bool}` (owned by F18), stored in +`PersistedSettings.experimental_features`, and read when a conversation +starts. A server inside a Docker runtime takes the user's switches from +`OH_RUNTIME_EXPERIMENTAL_FEATURES` instead, and its route stops checking the +switch because the host checks it before forwarding. + +Source: `openhands-tools/openhands/tools/ask_user/`, `openhands-sdk/openhands/sdk/event/ask_user.py`, `openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py` (`answer_ask`), `openhands-sdk/openhands/sdk/agent/agent.py` (`ends_turn`), `openhands-sdk/openhands/sdk/conversation/goal/controller.py` (`GoalAsked`), `openhands-agent-server/openhands/agent_server/event_router.py` (`ask_user_router`), `openhands-agent-server/openhands/agent_server/event_service.py` (`answer_ask`, goal loop), `openhands-agent-server/openhands/agent_server/models.py` (`AskUserAnswerRequest`), `openhands-agent-server/openhands/agent_server/experimental_features.py`, `openhands-agent-server/openhands/agent_server/tool_router.py`, `clients/typescript/src/client/conversation-client.ts`, `clients/typescript/src/events/types.ts` + +Needs: `llm` + +Routes: `POST /api/conversations/{conversation_id}/ask_user/answer` + +## Sub-features + +- `F40.flag-gates`: `GET /api/settings/experimental` lists `ask_user` ("Decision cards") off on a fresh server; with it off the answer route is 409 `Decision cards are turned off` and a new conversation has no `ask_user` tool; `PATCH .../ask_user` `{"enabled": true}` turns it on and a new conversation started without naming the tool has exactly `ask_user`. +- `F40.question-event`: on DeepSeek flash, an agent asked to let the user pick calls `ask_user`: the log and the socket carry an `ActionEvent` (`tool_name: ask_user`, an `AskUserAction` with the question and two options) and its `AskUserObservation` (`is_error: false`, `Your turn ends here`), and the conversation is `finished` with no `finish` call. +- `F40.answer-errors`: an `ask_id` that is not the pending question, an option past the last one and an answer with neither option nor words are 400; a negative `option_index` or a missing `ask_id` is 422; an unknown conversation is 404 and a missing key 401; switching the flag off refuses a valid answer at once (409) while the conversation keeps its tool; none of these saves anything and the question stays pending. +- `F40.answer`: `POST .../ask_user/answer` `{ask_id, option_index, text}` returns the saved `AskUserAnswerEvent` (`source: user`, `option_label`, the words, `parent_id` the question's observation); the user message `[Answer to your question: ""]` / `Option 2: