Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
cdf6ce2
fix(sdk): save the secret registry when update_secrets changes it
BSmick6 Sep 1, 2026
e2fc0f9
fix(acp): mask secrets added after session start in the live stream
enyst Oct 9, 2026
2f02f24
test(sdk): secrets reach base_state.json before update_secrets returns
claude Nov 11, 2026
c3a0fc1
verify-sdk: S04 shows secrets in base_state.json right after the call
claude Nov 11, 2026
af77263
verify-agent-server: three secret bullets are no longer known bugs
claude Nov 11, 2026
185ed3b
docs(acp): shorten the masker comment to the local invariant
claude Nov 11, 2026
44248b5
fix(sdk): a state update event prints its value masked
claude Nov 11, 2026
a895725
verify-agent-server: say when a change made during a step is saved
claude Nov 11, 2026
d34c74e
docs(sdk): say why update_secrets copies the registry, and what a rel…
claude Nov 11, 2026
46e6059
verify-agent-server: two map sentences still described secrets as uns…
claude Nov 11, 2026
8aa8496
fix(sdk): a load that cannot decrypt the state no longer saves it
claude Nov 11, 2026
2e2f797
verify: two map lines say what the secrets change did
claude Nov 11, 2026
c0fa0cd
fix(sdk): only a persisted agent's context secrets are checked for fa…
claude Nov 11, 2026
efe3070
Merge remote-tracking branch 'origin/main' into claude/openhands-sdk-…
claude Nov 13, 2026
129d64e
verify-agent-server: a change posted during a step is saved at once now
claude Nov 13, 2026
e0aafdb
tests: a secret posted while a step waits on the model is saved at once
claude Nov 14, 2026
bfa0862
Pin the shared masking cache across update_secrets, and fix two stale…
claude Nov 14, 2026
ad34192
Merge remote-tracking branch 'origin/main' into claude/openhands-sdk-…
claude Nov 14, 2026
6d9b169
Merge remote-tracking branch 'origin/main' into claude/openhands-sdk-…
claude Nov 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -414,19 +414,17 @@ EOF
```
The response lists the secret with its value masked; on disk it is
encrypted.
- **Start-time secret visible to GET (`F04.create-secrets-get`), known bug.**
- **Start-time secret visible to GET (`F04.create-secrets-get`).**
Read the conversation created in the previous bullet.

Requires: `F04.create-secrets`
```sh
control-agent-server api GET "/api/conversations/$SECID" --expect 200 \
--check secret_registry.secret_sources.QA_F04_TOKEN.kind eq StaticSecret --save F04.create-secrets-get/get # bug: status eq 200 && response.body.secret_registry.secret_sources exists && checks.0.actual eq null
--check secret_registry.secret_sources.QA_F04_TOKEN.kind eq StaticSecret --save F04.create-secrets-get/get
```
Expected: the GET lists `QA_F04_TOKEN` like the create response did. Today
it returns `{"secret_sources": {}}`: start-time secrets are applied to the
live state in place without an autosave, and GET reads the
`base_state.json` snapshot. The secret appears after the next state save (a
tags PATCH, a message), so the runtime itself still has it.
The GET lists `QA_F04_TOKEN` like the create response did:
`LocalConversation.update_secrets` assigns an updated registry to the
state, which saves `base_state.json` before the create returns.
- **Plain-string secret value (`F04.create-secrets-plain`).**
`POST .../secrets` accepts `{"NAME": "value"}`, but the create body takes
only secret objects, so a plain string there is refused with 422.
Expand Down Expand Up @@ -1094,9 +1092,9 @@ EOF
`.../events` with `"run": false`; `conversation start --prompt ... --no-run`
does exactly that.
- `GET`, `search` and the batch get read the autosaved `base_state.json`
snapshot (plus `meta.json`), not live memory. Changes applied in place
without an autosave (start-time secrets, `POST .../secrets`) stay invisible
until the next state save: see `F04.create-secrets-get`.
snapshot (plus `meta.json`), not live memory. A change applied in place
without an autosave would stay invisible until the next state save;
secrets are saved at once (`F04.create-secrets-get`).
- Secrets never come back: `agent.llm.api_key` is `null` in REST responses
(the masked value is re-parsed as "no secret") and `**********` in the
socket's `full_state` frame. Assert with `ne`, never by printing a key.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Routes: `POST /api/conversations/{conversation_id}/events/respond_to_confirmatio
- `F08.analyzer-set-clear`: `POST .../security_analyzer` sets `PatternSecurityAnalyzer` or `LLMSecurityAnalyzer` and clears it with `null`; GET, `base_state.json` and the socket reflect each change.
- `F08.analyzer-errors`: a missing `security_analyzer` field and an unknown kind are 422; an unknown conversation is 404; no key is 401; the stored analyzer is unchanged.
- `F08.confirm-risky`: under `ConfirmRisky` with `PatternSecurityAnalyzer`, a low-risk command runs unasked and `rm -rf` waits for confirmation; once the analyzer is cleared, an unrated command waits too.
- `F08.secrets-update`: `POST .../secrets` accepts plain strings, `{"value": ...}` and `StaticSecret` objects (200); once the state is saved, GET lists the names with masked values and `base_state.json` holds them encrypted, never in plaintext.
- `F08.secrets-update`: `POST .../secrets` accepts plain strings, `{"value": ...}` and `StaticSecret` objects (200); GET lists the names with masked values and `base_state.json` holds them encrypted, never in plaintext.
- `F08.secrets-visible`: a secret posted at runtime is listed by GET right away, without waiting for another state change.
- `F08.secrets-errors`: a missing or non-object `secrets`, an unknown secret kind and a `LookupSecret` without `url` are 422; an unknown conversation is 404; no key is 401; nothing is stored.
- `F08.secrets-bad-value`: a number, `null` or a boolean as a secret value is refused with 422, not a 500.
Expand All @@ -48,7 +48,7 @@ Routes: `POST /api/conversations/{conversation_id}/events/respond_to_confirmatio
- `F08.secrets-lookup-self`: a runtime `LookupSecret` whose URL is the server's own `/api/settings/secrets/<name>` (hostless, made absolute from `OH_INTERNAL_SERVER_URL`, or absolute on the run's address) is answered in process from the secrets store with no HTTP request and no session key; a name the store lacks falls through to HTTP (401 without a key) and the command gets no value.
- `F08.secrets-profile-scope`: for a conversation launched from an Agent Profile with `secret_refs`, names outside the list are silently dropped (200).
- `F08.secrets-no-cipher`: on a server with no cipher (no `OH_SECRET_KEY`, no session key) a runtime secret reaches commands while the conversation lives, is persisted only as the redaction placeholder (the server warns at startup), and is empty after a restart although GET still lists the name.
- `F08.secrets-restart`: a runtime secret that reached `base_state.json` is still injected into commands after a server restart.
- `F08.secrets-restart`: a runtime secret is still injected into commands after a server restart.
- `F08.secrets-unsaved-restart`: a runtime secret survives a restart even when nothing else changed in the conversation after it was posted.

## How to get to it (agent POV)
Expand Down Expand Up @@ -423,8 +423,8 @@ KID=$(control-agent-server api POST /api/conversations \
With the analyzer cleared, the next `echo` is unrated (`UNKNOWN`) and
`confirm_unknown` makes it wait; accepting it lets the agent finish.
- **Update secrets (`F08.secrets-update`).** The three accepted value forms,
on the idle conversation `$KID`. A tags PATCH forces a state save first
(see the next bullet for why).
on the idle conversation `$KID`. A tags PATCH also saves the state; GET
does not depend on it (`F08.secrets-visible`).
```sh
control-agent-server api POST "/api/conversations/$KID/secrets" \
--json '{"secrets": {"QA_F08_PLAIN": "qa-f08-plain-value", "QA_F08_DICT": {"value": "qa-f08-dict-value"}, "QA_F08_STATIC": {"kind": "StaticSecret", "value": "qa-f08-static-value", "description": "qa static"}}}' \
Expand All @@ -447,7 +447,7 @@ KID=$(control-agent-server api POST /api/conversations \
`base_state.json` holds cipher text: neither the plaintext nor the
`**********` placeholder a server without a cipher writes
(`F08.secrets-no-cipher`).
- **Runtime secret visible to GET (`F08.secrets-visible`), known bug.** The
- **Runtime secret visible to GET (`F08.secrets-visible`).** The
positive control first: a secret followed by a tags PATCH (which saves the
state) is listed by GET. Then one more secret is posted and the
conversation is read right away.
Expand All @@ -457,15 +457,12 @@ KID=$(control-agent-server api POST /api/conversations \
control-agent-server api GET "/api/conversations/$KID" --check secret_registry.secret_sources.QA_F08_VISIBLE_SAVED.kind eq StaticSecret
control-agent-server api POST "/api/conversations/$KID/secrets" --json '{"secrets": {"QA_F08_FRESH": "qa-f08-fresh-value"}}' --expect 200
control-agent-server api GET "/api/conversations/$KID" --check secret_registry.secret_sources.QA_F08_FRESH.kind eq StaticSecret \
--save F08.secrets-visible/get # bug: status eq 200 && checks.0.actual eq null && response.body.secret_registry.secret_sources.QA_F08_VISIBLE_SAVED.kind eq StaticSecret
--save F08.secrets-visible/get
```
Expected: GET lists `QA_F08_FRESH` as it lists the saved one. Today it does not:
`LocalConversation.update_secrets` mutates
`state.secret_registry.secret_sources` in place, the state only autosaves
on attribute assignment, and GET reads the `base_state.json` snapshot. The
name appears after the next state change (a PATCH, a message, a policy
change); until then the live registry has it but no reader can see it, and
a restart loses it (`F08.secrets-unsaved-restart`).
GET lists `QA_F08_FRESH` as it lists the saved one: the state only
autosaves on attribute assignment, so `LocalConversation.update_secrets`
assigns an updated copy of the registry, and `base_state.json` (which GET
reads) holds the name before the POST returns.
- **Secret errors (`F08.secrets-errors`).** Bad bodies, an unknown id, no key.
```sh
control-agent-server api POST "/api/conversations/$KID/secrets" --json '{}' --expect 422 --check detail.0.loc contains secrets \
Expand Down Expand Up @@ -760,7 +757,7 @@ KID=$(control-agent-server api POST /api/conversations \
```
After the restart the server is healthy and `QA_F08_TOKEN` (saved
encrypted with the run's `OH_SECRET_KEY`) is injected again.
- **Unsaved secret restart (`F08.secrets-unsaved-restart`), known bug.** One
- **Unsaved secret restart (`F08.secrets-unsaved-restart`).** One
secret is saved by a PATCH (the positive control), a second is posted with
nothing after it, then the server restarts.
```sh
Expand All @@ -773,13 +770,12 @@ KID=$(control-agent-server api POST /api/conversations \
control-agent-server api PATCH "/api/conversations/$UNS" --json '{"tags": {"qa": "f08-after-restart"}}' --expect 200
control-agent-server api GET "/api/conversations/$UNS" --check secret_registry.secret_sources.QA_F08_SAVED.kind eq StaticSecret
control-agent-server api GET "/api/conversations/$UNS" --check secret_registry.secret_sources.QA_F08_UNSAVED.kind eq StaticSecret \
--save F08.secrets-unsaved-restart/get # bug: status eq 200 && checks.0.actual eq null && response.body.secret_registry.secret_sources.QA_F08_SAVED.kind eq StaticSecret
--save F08.secrets-unsaved-restart/get
```
Expected: both names after the restart. Today `QA_F08_SAVED` is there and
`QA_F08_UNSAVED` is gone: the POST never reached `base_state.json` (see
`F08.secrets-visible`) and runtime secrets are not written to `meta.json`,
so a restart, an idle eviction (default 20 minutes) or a sandbox pause
right after `POST .../secrets` silently drops it.
Both names are there after the restart: the POST wrote `base_state.json`
before returning (see `F08.secrets-visible`), so a restart, an idle
eviction (default 20 minutes) or a sandbox pause right after
`POST .../secrets` keeps it.

## Gotchas

Expand Down Expand Up @@ -827,10 +823,12 @@ KID=$(control-agent-server api POST /api/conversations \
sub-agent started by the task tool gets the conversation's secrets under
the same rules (`F08.secrets-task-delegate`).
- Runtime secret changes made through `POST .../secrets` live only in
`base_state.json` and are never written to `meta.json`, so a runtime secret
survives a reload (restart, idle eviction, sandbox pause) only once a state
save has written it to `base_state.json`; one that never got there is
dropped. See `F08.secrets-restart` and `F08.secrets-unsaved-restart`.
`base_state.json` (written before the POST returns) and are never written
to `meta.json`. A runtime secret with a new name therefore survives a
reload (restart, idle eviction, sandbox pause); see `F08.secrets-restart`
and `F08.secrets-unsaved-restart`. A new value for a name the conversation
was created with does not: the load re-applies the creation-time copy that
`meta.json` keeps.
- Unknown conversations answer 404 `{"detail": "Not Found"}` on the
conversation routes and 404 `Conversation not found: <id>` on
`respond_to_confirmation` (an events route).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1104,10 +1104,10 @@ printf -- '---\nname: qa-f21-skill\ndescription: QA F21 user skill\n---\nQA_F21_
- A profile always launches its LLM with streaming on; the meta-profile
reference is only checked when `enable_classify_and_switch_llm_tool` is
true.
- Secrets given at conversation start are kept in memory and survive a
restart, but `GET /api/conversations/{id}` shows an empty
`secret_registry` until the conversation's state is saved again (a run,
a tag edit); the secret-scope bullet edits a tag first, as F08 does.
- Secrets given at conversation start are saved with the conversation, so
`GET /api/conversations/{id}` lists them at once
(`F04.create-secrets-get`). The secret-scope bullet still edits a tag
before it reads the registry; that step is no longer needed for the read.
- A profile file that is valid JSON but not a valid profile is listed
(summaries read raw JSON) yet is 400 on GET. Through other families'
routes the same profile is a deliberate 500 `Could not load agent
Expand Down
2 changes: 1 addition & 1 deletion .agents/skills/verify-agent-server/references/mapping.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ Then exactly four H2 sections, in this order:
Two label markers keep the map honest without turning it red forever:
**known bug** and **blocked**. A bullet whose bold label says **known
bug** (for example
`**Stale secrets after update (`F08.secrets-visible`), known bug #1234.**`)
`**Remote server behind a bearer token (`F19.test-remote-bearer`), known bug #6687.**`)
follows the known-bug contract below. A bullet whose label says **blocked**
names its prerequisite in prose, is reported as `blocked`, and is not
executed. Later bullets never depend on variables set in either kind.
Expand Down
16 changes: 15 additions & 1 deletion .agents/skills/verify-sdk/programs/S04/secret_env.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
"""Give a conversation secrets the way a program does, then let the agent use
them in terminal commands: `update_secrets`, `Conversation(secrets=...)`, a
`SecretSource` subclass, overwriting a key, masking in observations and in the
persisted events. The model is a scripted `TestLLM`; the terminal is real.
persisted events, and what `base_state.json` holds right after each call. The
model is a scripted `TestLLM`; the terminal is real.

Usage: secret_env.py
"""

import json
from pathlib import Path

from sdk_verify import emit, make_llm, run_paths

from openhands.sdk import Agent, Conversation, Tool
Expand All @@ -32,6 +36,13 @@ def term(command: str) -> dict:
return {"tool": "terminal", "args": {"command": command}}


def saved_secret_names(conversation) -> list[str]:
"""Secret names in the conversation's base_state.json on disk right now."""
state_file = Path(conversation.state.persistence_dir) / "base_state.json"
saved = json.loads(state_file.read_text())
return sorted(saved["secret_registry"]["secret_sources"])


def observations(conversation) -> list[str]:
return [
"".join(
Expand Down Expand Up @@ -67,6 +78,7 @@ def observations(conversation) -> list[str]:
conversation.update_secrets(
{"SECRET_TOKEN": "first-secret-value-123", "FUNCTION_TOKEN": CountingSource()}
)
out["saved_after_update"] = saved_secret_names(conversation)
conversation.send_message("Use the secrets.")
conversation.run()
obs = observations(conversation)
Expand Down Expand Up @@ -125,9 +137,11 @@ def observations(conversation) -> list[str]:
conversation3 = Conversation(
agent=Agent(llm=make_llm(script3), tools=[Tool(name=TerminalTool.name)]),
workspace=str(paths["workspace"]),
persistence_dir=str(BASE),
secrets={"API_KEY": "constructor-value-1", "SRC_KEY": CountingSource()},
visualizer=None,
)
out["saved_after_constructor"] = saved_secret_names(conversation3)
conversation3.send_message("Echo the keys.")
conversation3.run()
out["constructor"] = observations(conversation3)[0]
Expand Down
Loading
Loading