diff --git a/.agents/skills/verify-agent-server/references/feature-map/F04-conversation-lifecycle.md b/.agents/skills/verify-agent-server/references/feature-map/F04-conversation-lifecycle.md index 0ef3a7d84d..a2481e1ee4 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F04-conversation-lifecycle.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F04-conversation-lifecycle.md @@ -414,19 +414,17 @@ EOF ``` The response lists the secret with its value masked; on disk it is encrypted. -- **Start-time secret visible to GET (`F04.create-secrets-get`), known bug.** +- **Start-time secret visible to GET (`F04.create-secrets-get`).** Read the conversation created in the previous bullet. Requires: `F04.create-secrets` ```sh control-agent-server api GET "/api/conversations/$SECID" --expect 200 \ - --check secret_registry.secret_sources.QA_F04_TOKEN.kind eq StaticSecret --save F04.create-secrets-get/get # bug: status eq 200 && response.body.secret_registry.secret_sources exists && checks.0.actual eq null + --check secret_registry.secret_sources.QA_F04_TOKEN.kind eq StaticSecret --save F04.create-secrets-get/get ``` - Expected: the GET lists `QA_F04_TOKEN` like the create response did. Today - it returns `{"secret_sources": {}}`: start-time secrets are applied to the - live state in place without an autosave, and GET reads the - `base_state.json` snapshot. The secret appears after the next state save (a - tags PATCH, a message), so the runtime itself still has it. + The GET lists `QA_F04_TOKEN` like the create response did: + `LocalConversation.update_secrets` assigns an updated registry to the + state, which saves `base_state.json` before the create returns. - **Plain-string secret value (`F04.create-secrets-plain`).** `POST .../secrets` accepts `{"NAME": "value"}`, but the create body takes only secret objects, so a plain string there is refused with 422. @@ -1094,9 +1092,9 @@ EOF `.../events` with `"run": false`; `conversation start --prompt ... --no-run` does exactly that. - `GET`, `search` and the batch get read the autosaved `base_state.json` - snapshot (plus `meta.json`), not live memory. Changes applied in place - without an autosave (start-time secrets, `POST .../secrets`) stay invisible - until the next state save: see `F04.create-secrets-get`. + snapshot (plus `meta.json`), not live memory. A change applied in place + without an autosave would stay invisible until the next state save; + secrets are saved at once (`F04.create-secrets-get`). - Secrets never come back: `agent.llm.api_key` is `null` in REST responses (the masked value is re-parsed as "no secret") and `**********` in the socket's `full_state` frame. Assert with `ne`, never by printing a key. diff --git a/.agents/skills/verify-agent-server/references/feature-map/F08-confirmation-security-secrets.md b/.agents/skills/verify-agent-server/references/feature-map/F08-confirmation-security-secrets.md index 7c58e0ce6d..de31af3db3 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F08-confirmation-security-secrets.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F08-confirmation-security-secrets.md @@ -35,7 +35,7 @@ Routes: `POST /api/conversations/{conversation_id}/events/respond_to_confirmatio - `F08.analyzer-set-clear`: `POST .../security_analyzer` sets `PatternSecurityAnalyzer` or `LLMSecurityAnalyzer` and clears it with `null`; GET, `base_state.json` and the socket reflect each change. - `F08.analyzer-errors`: a missing `security_analyzer` field and an unknown kind are 422; an unknown conversation is 404; no key is 401; the stored analyzer is unchanged. - `F08.confirm-risky`: under `ConfirmRisky` with `PatternSecurityAnalyzer`, a low-risk command runs unasked and `rm -rf` waits for confirmation; once the analyzer is cleared, an unrated command waits too. -- `F08.secrets-update`: `POST .../secrets` accepts plain strings, `{"value": ...}` and `StaticSecret` objects (200); once the state is saved, GET lists the names with masked values and `base_state.json` holds them encrypted, never in plaintext. +- `F08.secrets-update`: `POST .../secrets` accepts plain strings, `{"value": ...}` and `StaticSecret` objects (200); GET lists the names with masked values and `base_state.json` holds them encrypted, never in plaintext. - `F08.secrets-visible`: a secret posted at runtime is listed by GET right away, without waiting for another state change. - `F08.secrets-errors`: a missing or non-object `secrets`, an unknown secret kind and a `LookupSecret` without `url` are 422; an unknown conversation is 404; no key is 401; nothing is stored. - `F08.secrets-bad-value`: a number, `null` or a boolean as a secret value is refused with 422, not a 500. @@ -48,7 +48,7 @@ Routes: `POST /api/conversations/{conversation_id}/events/respond_to_confirmatio - `F08.secrets-lookup-self`: a runtime `LookupSecret` whose URL is the server's own `/api/settings/secrets/` (hostless, made absolute from `OH_INTERNAL_SERVER_URL`, or absolute on the run's address) is answered in process from the secrets store with no HTTP request and no session key; a name the store lacks falls through to HTTP (401 without a key) and the command gets no value. - `F08.secrets-profile-scope`: for a conversation launched from an Agent Profile with `secret_refs`, names outside the list are silently dropped (200). - `F08.secrets-no-cipher`: on a server with no cipher (no `OH_SECRET_KEY`, no session key) a runtime secret reaches commands while the conversation lives, is persisted only as the redaction placeholder (the server warns at startup), and is empty after a restart although GET still lists the name. -- `F08.secrets-restart`: a runtime secret that reached `base_state.json` is still injected into commands after a server restart. +- `F08.secrets-restart`: a runtime secret is still injected into commands after a server restart. - `F08.secrets-unsaved-restart`: a runtime secret survives a restart even when nothing else changed in the conversation after it was posted. ## How to get to it (agent POV) @@ -423,8 +423,8 @@ KID=$(control-agent-server api POST /api/conversations \ With the analyzer cleared, the next `echo` is unrated (`UNKNOWN`) and `confirm_unknown` makes it wait; accepting it lets the agent finish. - **Update secrets (`F08.secrets-update`).** The three accepted value forms, - on the idle conversation `$KID`. A tags PATCH forces a state save first - (see the next bullet for why). + on the idle conversation `$KID`. A tags PATCH also saves the state; GET + does not depend on it (`F08.secrets-visible`). ```sh control-agent-server api POST "/api/conversations/$KID/secrets" \ --json '{"secrets": {"QA_F08_PLAIN": "qa-f08-plain-value", "QA_F08_DICT": {"value": "qa-f08-dict-value"}, "QA_F08_STATIC": {"kind": "StaticSecret", "value": "qa-f08-static-value", "description": "qa static"}}}' \ @@ -447,7 +447,7 @@ KID=$(control-agent-server api POST /api/conversations \ `base_state.json` holds cipher text: neither the plaintext nor the `**********` placeholder a server without a cipher writes (`F08.secrets-no-cipher`). -- **Runtime secret visible to GET (`F08.secrets-visible`), known bug.** The +- **Runtime secret visible to GET (`F08.secrets-visible`).** The positive control first: a secret followed by a tags PATCH (which saves the state) is listed by GET. Then one more secret is posted and the conversation is read right away. @@ -457,15 +457,12 @@ KID=$(control-agent-server api POST /api/conversations \ control-agent-server api GET "/api/conversations/$KID" --check secret_registry.secret_sources.QA_F08_VISIBLE_SAVED.kind eq StaticSecret control-agent-server api POST "/api/conversations/$KID/secrets" --json '{"secrets": {"QA_F08_FRESH": "qa-f08-fresh-value"}}' --expect 200 control-agent-server api GET "/api/conversations/$KID" --check secret_registry.secret_sources.QA_F08_FRESH.kind eq StaticSecret \ - --save F08.secrets-visible/get # bug: status eq 200 && checks.0.actual eq null && response.body.secret_registry.secret_sources.QA_F08_VISIBLE_SAVED.kind eq StaticSecret + --save F08.secrets-visible/get ``` - Expected: GET lists `QA_F08_FRESH` as it lists the saved one. Today it does not: - `LocalConversation.update_secrets` mutates - `state.secret_registry.secret_sources` in place, the state only autosaves - on attribute assignment, and GET reads the `base_state.json` snapshot. The - name appears after the next state change (a PATCH, a message, a policy - change); until then the live registry has it but no reader can see it, and - a restart loses it (`F08.secrets-unsaved-restart`). + GET lists `QA_F08_FRESH` as it lists the saved one: the state only + autosaves on attribute assignment, so `LocalConversation.update_secrets` + assigns an updated copy of the registry, and `base_state.json` (which GET + reads) holds the name before the POST returns. - **Secret errors (`F08.secrets-errors`).** Bad bodies, an unknown id, no key. ```sh control-agent-server api POST "/api/conversations/$KID/secrets" --json '{}' --expect 422 --check detail.0.loc contains secrets \ @@ -760,7 +757,7 @@ KID=$(control-agent-server api POST /api/conversations \ ``` After the restart the server is healthy and `QA_F08_TOKEN` (saved encrypted with the run's `OH_SECRET_KEY`) is injected again. -- **Unsaved secret restart (`F08.secrets-unsaved-restart`), known bug.** One +- **Unsaved secret restart (`F08.secrets-unsaved-restart`).** One secret is saved by a PATCH (the positive control), a second is posted with nothing after it, then the server restarts. ```sh @@ -773,13 +770,12 @@ KID=$(control-agent-server api POST /api/conversations \ control-agent-server api PATCH "/api/conversations/$UNS" --json '{"tags": {"qa": "f08-after-restart"}}' --expect 200 control-agent-server api GET "/api/conversations/$UNS" --check secret_registry.secret_sources.QA_F08_SAVED.kind eq StaticSecret control-agent-server api GET "/api/conversations/$UNS" --check secret_registry.secret_sources.QA_F08_UNSAVED.kind eq StaticSecret \ - --save F08.secrets-unsaved-restart/get # bug: status eq 200 && checks.0.actual eq null && response.body.secret_registry.secret_sources.QA_F08_SAVED.kind eq StaticSecret + --save F08.secrets-unsaved-restart/get ``` - Expected: both names after the restart. Today `QA_F08_SAVED` is there and - `QA_F08_UNSAVED` is gone: the POST never reached `base_state.json` (see - `F08.secrets-visible`) and runtime secrets are not written to `meta.json`, - so a restart, an idle eviction (default 20 minutes) or a sandbox pause - right after `POST .../secrets` silently drops it. + Both names are there after the restart: the POST wrote `base_state.json` + before returning (see `F08.secrets-visible`), so a restart, an idle + eviction (default 20 minutes) or a sandbox pause right after + `POST .../secrets` keeps it. ## Gotchas @@ -827,10 +823,12 @@ KID=$(control-agent-server api POST /api/conversations \ sub-agent started by the task tool gets the conversation's secrets under the same rules (`F08.secrets-task-delegate`). - Runtime secret changes made through `POST .../secrets` live only in - `base_state.json` and are never written to `meta.json`, so a runtime secret - survives a reload (restart, idle eviction, sandbox pause) only once a state - save has written it to `base_state.json`; one that never got there is - dropped. See `F08.secrets-restart` and `F08.secrets-unsaved-restart`. + `base_state.json` (written before the POST returns) and are never written + to `meta.json`. A runtime secret with a new name therefore survives a + reload (restart, idle eviction, sandbox pause); see `F08.secrets-restart` + and `F08.secrets-unsaved-restart`. A new value for a name the conversation + was created with does not: the load re-applies the creation-time copy that + `meta.json` keeps. - Unknown conversations answer 404 `{"detail": "Not Found"}` on the conversation routes and 404 `Conversation not found: ` on `respond_to_confirmation` (an events route). diff --git a/.agents/skills/verify-agent-server/references/feature-map/F21-agent-profiles.md b/.agents/skills/verify-agent-server/references/feature-map/F21-agent-profiles.md index 461dae47f1..b6a63dc3d2 100644 --- a/.agents/skills/verify-agent-server/references/feature-map/F21-agent-profiles.md +++ b/.agents/skills/verify-agent-server/references/feature-map/F21-agent-profiles.md @@ -1104,10 +1104,10 @@ printf -- '---\nname: qa-f21-skill\ndescription: QA F21 user skill\n---\nQA_F21_ - A profile always launches its LLM with streaming on; the meta-profile reference is only checked when `enable_classify_and_switch_llm_tool` is true. -- Secrets given at conversation start are kept in memory and survive a - restart, but `GET /api/conversations/{id}` shows an empty - `secret_registry` until the conversation's state is saved again (a run, - a tag edit); the secret-scope bullet edits a tag first, as F08 does. +- Secrets given at conversation start are saved with the conversation, so + `GET /api/conversations/{id}` lists them at once + (`F04.create-secrets-get`). The secret-scope bullet still edits a tag + before it reads the registry; that step is no longer needed for the read. - A profile file that is valid JSON but not a valid profile is listed (summaries read raw JSON) yet is 400 on GET. Through other families' routes the same profile is a deliberate 500 `Could not load agent diff --git a/.agents/skills/verify-agent-server/references/mapping.md b/.agents/skills/verify-agent-server/references/mapping.md index 4fffd2420c..201ecb302b 100644 --- a/.agents/skills/verify-agent-server/references/mapping.md +++ b/.agents/skills/verify-agent-server/references/mapping.md @@ -180,7 +180,7 @@ Then exactly four H2 sections, in this order: Two label markers keep the map honest without turning it red forever: **known bug** and **blocked**. A bullet whose bold label says **known bug** (for example - `**Stale secrets after update (`F08.secrets-visible`), known bug #1234.**`) + `**Remote server behind a bearer token (`F19.test-remote-bearer`), known bug #6687.**`) follows the known-bug contract below. A bullet whose label says **blocked** names its prerequisite in prose, is reported as `blocked`, and is not executed. Later bullets never depend on variables set in either kind. diff --git a/.agents/skills/verify-sdk/programs/S04/secret_env.py b/.agents/skills/verify-sdk/programs/S04/secret_env.py index 369ab7d7f0..e418407635 100644 --- a/.agents/skills/verify-sdk/programs/S04/secret_env.py +++ b/.agents/skills/verify-sdk/programs/S04/secret_env.py @@ -1,11 +1,15 @@ """Give a conversation secrets the way a program does, then let the agent use them in terminal commands: `update_secrets`, `Conversation(secrets=...)`, a `SecretSource` subclass, overwriting a key, masking in observations and in the -persisted events. The model is a scripted `TestLLM`; the terminal is real. +persisted events, and what `base_state.json` holds right after each call. The +model is a scripted `TestLLM`; the terminal is real. Usage: secret_env.py """ +import json +from pathlib import Path + from sdk_verify import emit, make_llm, run_paths from openhands.sdk import Agent, Conversation, Tool @@ -32,6 +36,13 @@ def term(command: str) -> dict: return {"tool": "terminal", "args": {"command": command}} +def saved_secret_names(conversation) -> list[str]: + """Secret names in the conversation's base_state.json on disk right now.""" + state_file = Path(conversation.state.persistence_dir) / "base_state.json" + saved = json.loads(state_file.read_text()) + return sorted(saved["secret_registry"]["secret_sources"]) + + def observations(conversation) -> list[str]: return [ "".join( @@ -67,6 +78,7 @@ def observations(conversation) -> list[str]: conversation.update_secrets( {"SECRET_TOKEN": "first-secret-value-123", "FUNCTION_TOKEN": CountingSource()} ) +out["saved_after_update"] = saved_secret_names(conversation) conversation.send_message("Use the secrets.") conversation.run() obs = observations(conversation) @@ -125,9 +137,11 @@ def observations(conversation) -> list[str]: conversation3 = Conversation( agent=Agent(llm=make_llm(script3), tools=[Tool(name=TerminalTool.name)]), workspace=str(paths["workspace"]), + persistence_dir=str(BASE), secrets={"API_KEY": "constructor-value-1", "SRC_KEY": CountingSource()}, visualizer=None, ) +out["saved_after_constructor"] = saved_secret_names(conversation3) conversation3.send_message("Echo the keys.") conversation3.run() out["constructor"] = observations(conversation3)[0] diff --git a/.agents/skills/verify-sdk/references/feature-map/S04-local-conversation.md b/.agents/skills/verify-sdk/references/feature-map/S04-local-conversation.md index 54982621d5..6aac84668b 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S04-local-conversation.md +++ b/.agents/skills/verify-sdk/references/feature-map/S04-local-conversation.md @@ -26,8 +26,8 @@ Examples: `examples/01_standalone_sdk/06_interactive_terminal_w_reasoning.py`, ` - `S04.arun-interrupt`: `await conversation.arun()` uses the async LLM path; `interrupt()` (safe from a signal handler or another thread) cancels it at once, even mid-LLM-call, sets PAUSED, emits an `InterruptEvent`, does not raise `CancelledError`, and the conversation can be resumed; with no `arun()` in flight it falls back to `pause()`. Unit: `tests/sdk/conversation/test_interrupt.py`. Example: `examples/01_standalone_sdk/50_async_cancellation.py`. Real-LLM: none. - `S04.arun-pause-persisted`: with a `persistence_dir`, a `pause()` made from another thread while `arun()` waits on the model returns at once and is in `base_state.json` (`execution_status: paused`) at once, before the model call returns; `arun()` then stops PAUSED after that call and a later `run()` finishes. Unit: `tests/sdk/agent/test_astep_releases_state_lock.py::test_pause_during_llm_call_is_persisted_at_once`. Example: none. Real-LLM: none. - `S04.cancel-token`: each `run()`/`arun()` gets a fresh `CancellationToken` that `interrupt()` sets before cancelling, so pending parallel tool calls are skipped. Unit: `tests/sdk/conversation/test_interrupt.py::test_interrupt_sets_cancel_token`, `::test_execute_batch_skips_all_on_pre_cancelled_token`. Example: `examples/01_standalone_sdk/50_async_cancellation.py`. Real-LLM: none. -- `S04.secrets-update`: `update_secrets({...})` accepts plain strings and `SecretSource` objects (resolved on first use, which is the first command output the conversation masks, then cached), overwrites existing keys, and exports referenced keys as environment variables for terminal commands. Unit: `tests/sdk/conversation/test_secrets_manager.py`, `tests/sdk/conversation/test_secret_source.py`. Example: `examples/01_standalone_sdk/12_custom_secrets.py`. Real-LLM: none. -- `S04.secrets-constructor`: `Conversation(secrets={...})` seeds the same registry at construction, for static and callable values. Unit: `tests/sdk/conversation/test_conversation_secrets_constructor.py`. Example: none. Real-LLM: none. +- `S04.secrets-update`: `update_secrets({...})` accepts plain strings and `SecretSource` objects (resolved on first use, which is the first command output the conversation masks, then cached), overwrites existing keys, exports referenced keys as environment variables for terminal commands, and saves the registry to `base_state.json` before it returns; a call made on the run loop's own thread inside a step (an event callback, or a tool under a synchronous `run()`) is written only when the step ends. Unit: `tests/sdk/conversation/test_secrets_manager.py`, `tests/sdk/conversation/test_secret_source.py`, `tests/sdk/conversation/test_conversation_secrets_constructor.py`. Example: `examples/01_standalone_sdk/12_custom_secrets.py`. Real-LLM: none. +- `S04.secrets-constructor`: `Conversation(secrets={...})` seeds the same registry at construction, for static and callable values, and the new conversation's `base_state.json` already holds them. Unit: `tests/sdk/conversation/test_conversation_secrets_constructor.py`. Example: none. Real-LLM: none. - `S04.secrets-masking`: secret values echoed by a command are replaced by `` in observations and never reach the persisted events. Unit: `tests/tools/terminal/test_secrets_masking.py`, `tests/cross/test_agent_secrets_integration.py`. Example: `examples/01_standalone_sdk/12_custom_secrets.py`. Real-LLM: none. - `S04.visualizer-default`: by default a `DefaultConversationVisualizer` prints each event as a Rich panel; `visualizer=None` turns it off; a class or instance can be passed. Unit: `tests/sdk/conversation/local/test_conversation_visualize_param.py`. Example: `examples/01_standalone_sdk/10_persistence.py`. Real-LLM: none. - `S04.visualizer-custom`: a `ConversationVisualizerBase` subclass passed as `visualizer=` receives every event through `on_event`; each event type has its own `visualize` rendering. Unit: `tests/sdk/conversation/test_visualizer.py`. Example: `examples/01_standalone_sdk/26_custom_visualizer.py`. Real-LLM: none. @@ -228,9 +228,11 @@ Scripted bullets use the SDK's `TestLLM` (`--script`, or `make_llm(script=...)` --check rotated eq "rotated=" --check length_after_rotation eq 25 \ --check constructor eq "api= src=" \ --check values_in_observations eq false --check values_on_disk eq false --check names_on_disk eq true \ + --check saved_after_update.0 eq FUNCTION_TOKEN --check saved_after_update.1 eq SECRET_TOKEN --check saved_after_update len-eq 2 \ + --check saved_after_constructor.0 eq API_KEY --check saved_after_constructor.1 eq SRC_KEY \ --check lookups_at_each_observation.0 eq 1 --check plain_callable contains "Invalid SecretValue" ``` - The lengths (22, then 25 after the overwrite) prove the real values reached the shell while the text the agent saw stayed masked. + The lengths (22, then 25 after the overwrite) prove the real values reached the shell while the text the agent saw stayed masked. `saved_after_update` and `saved_after_constructor` are the secret names read from `base_state.json` right after the call, before any message or run: a restart at that point keeps them. - **Visualizers (`S04.visualizer-default`, `S04.visualizer-custom`).** The default prints Rich panels (with and without passing it), `None` prints nothing, an instance with `skip_user_messages=True` drops the user panel, and a `ConversationVisualizerBase` subclass receives every event. ```sh export SDK_VERIFY_RUN=$(control-sdk new --field run) diff --git a/.agents/skills/verify-sdk/references/feature-map/S21-acp-agent.md b/.agents/skills/verify-sdk/references/feature-map/S21-acp-agent.md index e88d8b5c50..209c7986e2 100644 --- a/.agents/skills/verify-sdk/references/feature-map/S21-acp-agent.md +++ b/.agents/skills/verify-sdk/references/feature-map/S21-acp-agent.md @@ -33,7 +33,7 @@ Examples: `examples/01_standalone_sdk/40_acp_agent_example.py` - `S21.mcp-forwarding`: the agent's `mcp_config` is converted to ACP stdio or HTTP/SSE servers (with headers) and passed to `new_session` and `load_session`; none gives an empty list. Unit: `tests/sdk/agent/test_acp_agent.py::TestMcpConfigToAcpServers`, `::TestACPMcpForwarding`. Example: none. Real-LLM: none. - `S21.secrets-env`: conversation secrets and static secrets are injected into the subprocess env (empty or None values skipped); inherited npm env vars are stripped; env vars that conflict with the chosen auth are suppressed. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPSecretsEnvInjection`, `::TestACPSecretRegistryEnvInjection`, `::TestACPEnvConflictSuppression`. Example: none. Real-LLM: none. - `S21.file-secrets`: file-content credentials (`CODEX_AUTH_JSON`, `GOOGLE_APPLICATION_CREDENTIALS_JSON`, `KIMI_CODE_CONFIG_TOML`, `PI_AUTH_JSON`) are written to disk for the matching provider only, the env var pointed at them, rotated files written back and masked, and `acp_isolate_data_dir=True` gives the CLI a per-conversation data dir under `/acp/`. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPFileSecretMaterialisation`, `::TestACPFileSecretProviderScoping`, `::TestACPDataDirIsolation`, `tests/sdk/agent/test_acp_file_credentials.py`. Example: none. Real-LLM: none. -- `S21.masking`: secret values are masked in relayed message and thought chunks, tool-call fields and terminal output, fork text and the persisted turn; a masking failure fails safe. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPBridgeMasking`, `::TestMaskJsonValue`, `::TestACPStepMasksPersistedTurn`. Example: none. Real-LLM: none. +- `S21.masking`: secret values are masked in relayed message and thought chunks, tool-call fields and terminal output, fork text and the persisted turn, also for a secret added with `update_secrets` after the session started; a masking failure fails safe. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPBridgeMasking`, `::TestMaskJsonValue`, `::TestACPStepMasksPersistedTurn`, `::test_live_token_masking_follows_update_secrets_reassignment`. Example: none. Real-LLM: none. - `S21.tracing`: with observability on, each turn emits an `LLM` span whose output is the assistant message and child TOOL spans with output and the ACP call id, all marked ACP with the server name; the prompt is capped and masked (dropped if masking fails); tracing is inert when disabled and a broken backend never breaks the turn. Unit: `tests/sdk/agent/test_acp_tracing.py`. Example: none. Real-LLM: none. - `S21.cleanup`: `close()` terminates the subprocess with a bounded kill wait, closes the executor, is idempotent and error-tolerant, and an atexit hook (weak, not retaining the agent) cleans up forgotten agents. Unit: `tests/sdk/agent/test_acp_agent.py::TestACPAgentCleanup`, `::TestFilterJsonrpcLines`. Example: `examples/01_standalone_sdk/40_acp_agent_example.py` (calls `agent.close()`). Real-LLM: none. - `S21.live-turn`: a full conversation against a real ACP server (text turn, image turn, `ask_agent`) produces a reply and a non-zero cost. Unit: **none (untested)**. Example: `examples/01_standalone_sdk/40_acp_agent_example.py`. Real-LLM: none (only the example on the `test-examples` label). diff --git a/openhands-sdk/openhands/sdk/agent/acp_agent.py b/openhands-sdk/openhands/sdk/agent/acp_agent.py index a0df7314bc..5689097d53 100644 --- a/openhands-sdk/openhands/sdk/agent/acp_agent.py +++ b/openhands-sdk/openhands/sdk/agent/acp_agent.py @@ -1439,8 +1439,8 @@ def __init__(self) -> None: # signal that the ACP subprocess is still actively working. Set by # ACPAgent.step() to keep the agent-server's idle timer alive. self.on_activity: Any = None # Callable[[], None] | None - # Secret masker — set per turn by ACPAgent to - # ``state.secret_registry.mask_secrets_in_output``. Applied to streamed + # Secret masker — bound once by ACPAgent when it starts the ACP server; + # it masks with the state's current secret registry. Applied to streamed # text chunks and tool-call raw_input/raw_output/content before they # reach ``on_token`` / ``on_event`` so a subprocess that echoes an # injected credential never lands in the (persisted, network-relayed) @@ -3156,14 +3156,19 @@ def _start_acp_server(self, state: ConversationState) -> None: """Start the ACP subprocess and initialize the session.""" client = _OpenHandsACPBridge() self._client = client - # Bind the secret masker for the conversation's lifetime. It's derived - # from state.secret_registry (stable for the conversation) and touches - # only that registry, so it has none of the cross-thread/state-lock + + # Bind the secret masker for the conversation's lifetime. It touches + # only state.secret_registry, so it has none of the cross-thread/state-lock # hazards that make on_event/on_token strictly per-turn. Binding it here # (rather than per-turn in _reset_client_for_turn) keeps it available for # session updates AND for ask_agent() forks, which run on the shared # client and may fire while no step()/astep() turn is active. - client.mask = state.secret_registry.mask_secrets_in_output + # The registry is looked up on every call: the state's registry object + # is replaced when secrets change. + def _mask_with_current_registry(text: str) -> str: + return state.secret_registry.mask_secrets_in_output(text) + + client.mask = _mask_with_current_registry self._bind_file_credential_masking() # Build the subprocess environment. Precedence, highest first: diff --git a/openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py b/openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py index 7cefe56515..7d79561d61 100644 --- a/openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py +++ b/openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py @@ -142,7 +142,7 @@ from openhands.sdk.tool.builtins import InvokeSkillTool from openhands.sdk.tool.client_tool import ClientToolSpec from openhands.sdk.tool.schema import Action, Observation -from openhands.sdk.utils.cipher import Cipher +from openhands.sdk.utils.cipher import FERNET_TOKEN_PREFIX, Cipher from openhands.sdk.utils.pydantic_secrets import client_context from openhands.sdk.utils.redact import ( redact_text_secrets, @@ -248,6 +248,15 @@ def _same_llm_config(a: LLM, b: LLM) -> bool: ) +def _is_undecrypted(secret: SecretValue) -> bool: + """True for what a load under a key that cannot decrypt the state leaves + in the persisted agent's context secrets: cipher text kept as a string, + or a ``StaticSecret`` without its value.""" + if isinstance(secret, str): + return secret.startswith(FERNET_TOKEN_PREFIX) + return isinstance(secret, StaticSecret) and secret.value is None + + class LocalConversation(BaseConversation): agent: AgentBase workspace: LocalWorkspace @@ -500,6 +509,7 @@ def __init__( # base_state.json is the source of truth for the agent. On resume with # ``agent=None`` the state holds the persisted agent; adopt it here so # ``self.agent`` and ``self._state.agent`` are the same object. + persisted_agent_adopted = agent is None if agent is None: agent = self._state.agent # The persisted agent carries client-tool ``Tool`` specs, but in a @@ -655,6 +665,11 @@ def _flush_pending_reactions(e): existing_sources = self._state.secret_registry.secret_sources fill_secrets: dict[str, SecretValue] = {} for name, secret in ctx_secrets.items(): + if persisted_agent_adopted and _is_undecrypted(secret): + # Filling it in would save the state, and with it + # every other value this load could not decrypt. A + # caller's own agent is taken as given. + continue existing = existing_sources.get(name) # Refill only when absent, or when a StaticSecret lost its # value to redacted/no-cipher serialization (value is None). @@ -3575,7 +3590,17 @@ def update_secrets(self, secrets: Mapping[str, SecretValue]) -> None: when a command references the secret key. """ with self._state: - self._state.secret_registry.update_secrets(secrets) + # The state saves itself when a field is assigned, not when one is + # changed in place, so the registry is replaced by an updated copy. + # model_copy keeps the masking cache shared with the previous + # object, which ACP credential and stream masking may still hold. + registry = self._state.secret_registry.model_copy( + update={ + "secret_sources": dict(self._state.secret_registry.secret_sources) + } + ) + registry.update_secrets(secrets) + self._state.secret_registry = registry logger.info(f"Added {len(secrets)} secrets to conversation") def set_security_analyzer(self, analyzer: SecurityAnalyzerBase | None) -> None: diff --git a/openhands-sdk/openhands/sdk/event/conversation_state.py b/openhands-sdk/openhands/sdk/event/conversation_state.py index caa968cb10..f483cc91f2 100644 --- a/openhands-sdk/openhands/sdk/event/conversation_state.py +++ b/openhands-sdk/openhands/sdk/event/conversation_state.py @@ -103,4 +103,6 @@ def from_conversation_state( return cls(key=FULL_STATE_KEY, value=state_snapshot) def __str__(self) -> str: - return f"ConversationStateUpdate(key={self.key}, value={self.value})" + # Through the serializer, so a secret in the value prints masked. + value = self.model_dump(mode="json")["value"] + return f"ConversationStateUpdate(key={self.key}, value={value})" diff --git a/tests/sdk/agent/test_acp_agent.py b/tests/sdk/agent/test_acp_agent.py index 26c35827ad..3a54f54755 100644 --- a/tests/sdk/agent/test_acp_agent.py +++ b/tests/sdk/agent/test_acp_agent.py @@ -10492,3 +10492,49 @@ def test_still_warns_when_nothing_is_configured(self, tmp_path, caplog): assert "session creation may fail" in caplog.text assert "GEMINI_API_KEY is unset" in caplog.text + + +@pytest.mark.asyncio +async def test_live_token_masking_follows_update_secrets_reassignment(tmp_path): + """Secrets added after ACP session start must not leak through on_token. + + ``update_secrets`` reassigns ``state.secret_registry`` to a fresh object, so + the ACP masker bound in ``_start_acp_server`` must read the current registry + on every call rather than capturing a bound method of the original registry; + otherwise a newly-added secret streams in cleartext through the live + ``on_token`` relay even though the persisted event is masked. + """ + from acp.schema import AgentMessageChunk, TextContentBlock + + agent = _make_agent() + state = _make_state(tmp_path) + conn = TestACPSessionIdPersistence._make_conn() + try: + TestACPSessionIdPersistence._patched_start_acp_server(agent, state, conn=conn) + + bridge = agent._client + assert bridge is not None + streamed: list[str] = [] + bridge.on_token = streamed.append + + # Replace the registry, as a secrets update does. + new_registry = state.secret_registry.model_copy( + update={"secret_sources": dict(state.secret_registry.secret_sources)} + ) + new_registry.update_secrets({"MY_TOKEN": "sk-NEWLYADDED-123456"}) + with state: + state.secret_registry = new_registry + + chunk = MagicMock(spec=AgentMessageChunk) + chunk.content = MagicMock(spec=TextContentBlock) + chunk.content.text = "token streamed: sk-NEWLYADDED-123456 tail" + await bridge.session_update("sess-new", chunk) + + assert streamed, "on_token should have received the streamed chunk" + assert "sk-NEWLYADDED-123456" not in streamed[-1] + assert "" in streamed[-1] + finally: + agent._unregister_atexit_cleanup() + if agent._executor is not None: + agent._executor.close() + agent._executor = None diff --git a/tests/sdk/agent/test_astep_releases_state_lock.py b/tests/sdk/agent/test_astep_releases_state_lock.py index 7bfb1e4f02..c90db69cc7 100644 --- a/tests/sdk/agent/test_astep_releases_state_lock.py +++ b/tests/sdk/agent/test_astep_releases_state_lock.py @@ -159,3 +159,45 @@ async def test_pause_during_llm_call_is_persisted_at_once(tmp_path): assert llm._probe["save_depth_during_call"] == 0 saved = json.loads(convo._state._fs.read(BASE_STATE)) assert saved["execution_status"] == ConversationExecutionStatus.PAUSED.value + + +class _SecretPostingLLM(TestLLM): + """Adds a secret from another thread mid-``acompletion`` and records the + secret names ``base_state.json`` holds right after that call returns.""" + + _convo_box: list = PrivateAttr(default_factory=list) + _probe: dict = PrivateAttr(default_factory=dict) + + async def acompletion(self, *args: Any, **kwargs: Any) -> LLMResponse: + convo = self._convo_box[0] + # The server's POST .../secrets handler calls update_secrets() from + # another thread. + await asyncio.to_thread(convo.update_secrets, {"POSTED_MID_STEP": "value"}) + saved = json.loads(convo._state._fs.read(BASE_STATE)) + self._probe["saved_names"] = sorted(saved["secret_registry"]["secret_sources"]) + return await super().acompletion(*args, **kwargs) + + +@pytest.mark.asyncio +async def test_secret_posted_during_llm_call_is_persisted_at_once(tmp_path): + """A secret added while the step has released the lock for the network + call is in base_state.json when update_secrets() returns, not at the end + of the step: a restart in that window must not lose it.""" + llm = _SecretPostingLLM.from_messages([LLMContentPolicyViolationError()]) + assert isinstance(llm, _SecretPostingLLM) + agent = Agent(llm=llm, tools=[]) + convo = Conversation( + agent=agent, persistence_dir=str(tmp_path), workspace=str(tmp_path) + ) + assert isinstance(convo, LocalConversation) + convo._ensure_agent_ready() + llm._convo_box.append(convo) + convo.send_message("hello") + + # Nothing else is waiting to be saved, so only the secret's own save can + # put it in the file. + convo._step_holds_state_lock = True + with convo._state: + await agent.astep(convo, on_event=[].append) + + assert llm._probe["saved_names"] == ["POSTED_MID_STEP"] diff --git a/tests/sdk/conversation/test_conversation_secrets_constructor.py b/tests/sdk/conversation/test_conversation_secrets_constructor.py index 0dbdc37d84..e38e3dcd60 100644 --- a/tests/sdk/conversation/test_conversation_secrets_constructor.py +++ b/tests/sdk/conversation/test_conversation_secrets_constructor.py @@ -1,17 +1,23 @@ """Tests for Conversation constructor with secrets parameter.""" +import json import tempfile +from pathlib import Path from unittest.mock import patch import pytest from pydantic import SecretStr from openhands.sdk.agent import Agent +from openhands.sdk.context import AgentContext from openhands.sdk.conversation import Conversation from openhands.sdk.conversation.impl.local_conversation import LocalConversation from openhands.sdk.conversation.impl.remote_conversation import RemoteConversation +from openhands.sdk.conversation.persistence_const import BASE_STATE +from openhands.sdk.event.conversation_state import ConversationStateUpdateEvent from openhands.sdk.llm import LLM -from openhands.sdk.secret import SecretSource +from openhands.sdk.secret import LookupSecret, SecretSource +from openhands.sdk.utils.cipher import FERNET_TOKEN_PREFIX, Cipher from openhands.sdk.workspace import RemoteWorkspace from .conftest import create_mock_http_client @@ -345,3 +351,175 @@ def test_secrets_parameter_type_validation(): with tempfile.TemporaryDirectory() as tmpdir: conv = Conversation(agent=agent, workspace=tmpdir, secrets=None) assert isinstance(conv, LocalConversation) + + +def _persisted_secret_names(persistence_dir: str) -> set[str]: + """Secret names stored in base_state.json on disk, without a later save.""" + [base_state] = Path(persistence_dir).rglob(BASE_STATE) + data = json.loads(base_state.read_text()) + return set(data["secret_registry"]["secret_sources"]) + + +def test_update_secrets_saves_base_state_before_returning(): + """Start-time and runtime secrets reach base_state.json right away (#4810).""" + agent = create_test_agent() + cipher = Cipher("test-secret-key") + with tempfile.TemporaryDirectory() as tmpdir: + conv = LocalConversation( + agent=agent, + workspace=tmpdir, + persistence_dir=tmpdir, + secrets={"START_TOKEN": "start-value"}, + cipher=cipher, + visualizer=None, + ) + assert _persisted_secret_names(tmpdir) == {"START_TOKEN"} + + conv.update_secrets({"RUNTIME_TOKEN": "runtime-value"}) + assert _persisted_secret_names(tmpdir) == {"START_TOKEN", "RUNTIME_TOKEN"} + + conv.update_secrets({"START_TOKEN": "overridden"}) + conv_id = conv.id + conv.close() + + resumed = LocalConversation( + agent=agent, + workspace=tmpdir, + persistence_dir=tmpdir, + conversation_id=conv_id, + cipher=cipher, + visualizer=None, + ) + registry = resumed.state.secret_registry + assert registry.get_secrets_as_env_vars("$START_TOKEN $RUNTIME_TOKEN") == { + "START_TOKEN": "overridden", + "RUNTIME_TOKEN": "runtime-value", + } + resumed.close() + + +def test_update_secrets_keeps_masking_shared_with_a_registry_held_from_before(): + """A step's stream masker and the ACP credential lifecycle keep the registry + they started with: a value either side learns after an update is masked by + the other.""" + agent = create_test_agent() + with tempfile.TemporaryDirectory() as tmpdir: + conv = LocalConversation(agent=agent, workspace=tmpdir, visualizer=None) + held = conv.state.secret_registry + held_stream_mask = held.compile_stream_mask + + conv.update_secrets({"RUNTIME_TOKEN": "runtime-value"}) + current = conv.state.secret_registry + + current.get_secrets_as_env_vars("echo $RUNTIME_TOKEN") + masker = held_stream_mask() + assert masker.feed("leak: runtime-value") + masker.flush() == ( + "leak: " + ) + + held.track_exported_values({"ROTATED_FILE": "rotated-file-value"}) + assert current.mask_secrets_in_output("leak: rotated-file-value") == ( + "leak: " + ) + conv.close() + + +def test_load_under_a_key_that_cannot_decrypt_leaves_the_saved_state_alone(): + """A conversation opened with the wrong key shows no credentials, and it + must not save that: once the right key is back, the agent's key and its + context secret are too.""" + agent = Agent( + llm=LLM(model="gpt-4o-mini", api_key=SecretStr("agent-key"), usage_id="agent"), + tools=[], + agent_context=AgentContext(secrets={"CTX_TOKEN": "ctx-value"}), + ) + + with tempfile.TemporaryDirectory() as tmpdir: + + def reopen(key: str, conversation_id=None, agent=None) -> LocalConversation: + return LocalConversation( + agent=agent, + workspace=tmpdir, + persistence_dir=tmpdir, + conversation_id=conversation_id, + cipher=Cipher(key), + visualizer=None, + ) + + created = reopen("right-key", agent=agent) + conversation_id = created.id + created.close() + + reopen("another-key", conversation_id).close() + + restored = reopen("right-key", conversation_id) + api_key = restored.agent.llm.api_key + assert isinstance(api_key, SecretStr) + assert api_key.get_secret_value() == "agent-key" + registry = restored.state.secret_registry + assert registry.get_secrets_as_env_vars("$CTX_TOKEN") == { + "CTX_TOKEN": "ctx-value" + } + restored.close() + + +def test_callers_own_context_secret_is_registered_whatever_it_looks_like(): + """Only a persisted agent can carry values that failed to decrypt. A secret + on the agent the caller passes is taken as given, also one that happens + to start like cipher text.""" + token = FERNET_TOKEN_PREFIX + "-a-token-the-user-supplied" + agent = Agent( + llm=LLM(model="gpt-4o-mini", api_key=SecretStr("agent-key"), usage_id="agent"), + tools=[], + agent_context=AgentContext(secrets={"TOKEN": token}), + ) + + with tempfile.TemporaryDirectory() as tmpdir: + conversation = LocalConversation(agent=agent, workspace=tmpdir, visualizer=None) + registry = conversation.state.secret_registry + assert registry.get_secrets_as_env_vars("$TOKEN") == {"TOKEN": token} + conversation.close() + + +@pytest.mark.parametrize( + "cipher", [Cipher("test-secret-key"), None], ids=["cipher", "no-cipher"] +) +def test_secrets_state_update_event_never_carries_a_secret_value(cipher): + """A secrets change is announced as one state update with names and masked + values: in the event, in its printed form and in the saved event file.""" + agent = create_test_agent() + secrets = { + "TOKEN": "plain-secret-value", + "LOOKUP": LookupSecret( + url="http://127.0.0.1:9/secret", + headers={"Authorization": "Bearer header-secret-value"}, + ), + } + with tempfile.TemporaryDirectory() as tmpdir: + conv = LocalConversation( + agent=agent, + workspace=tmpdir, + persistence_dir=tmpdir, + cipher=cipher, + visualizer=None, + ) + # The agent-server publishes state changes as conversation events. + conv.state.set_on_state_change(conv._on_event) + conv.update_secrets(secrets) + conv.update_secrets(secrets) + + updates = [ + event + for event in conv.state.events + if isinstance(event, ConversationStateUpdateEvent) + and event.key == "secret_registry" + ] + assert len(updates) == 1 + event_json = updates[0].model_dump_json() + assert "TOKEN" in event_json + assert FERNET_TOKEN_PREFIX not in event_json + saved = "".join(p.read_text() for p in Path(tmpdir).rglob("*.json")) + for text in (event_json, str(updates[0]), saved): + assert "plain-secret-value" not in text + assert "header-secret-value" not in text + conv.close()