Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
5383a6f
fix(sdk): keep the source's cipher and secrets on a forked conversation
claude Oct 13, 2026
0fb50d6
test(agent-server): preserve fork credentials across restart
kyletser Oct 3, 2026
e918141
Merge upstream main and reconcile verification recipes
enyst Oct 26, 2026
0308e03
Merge main into PR #5765
enyst Oct 26, 2026
4e96b98
Merge main and retain fork credentials and budget coverage
enyst Nov 7, 2026
7161565
Merge #5765 (fork keeps the source's cipher and secrets)
claude Nov 11, 2026
a50176d
fix(sdk): copy confirmation_policy and security_analyzer in LocalConv…
BSmick6 Sep 4, 2026
1741208
fix(conversation): a fork gets its own security analyzer, credentials…
enyst Oct 9, 2026
8bc2352
test(sdk): a fork keeps the source's confirmation policy
claude Nov 11, 2026
214ccba
verify-sdk: S04 shows what a fork keeps of the source's state
claude Nov 11, 2026
e64a60d
fix(sdk): fork copies its values the same way, before the fork exists
claude Nov 11, 2026
e27a26e
fix(sdk): a fork keeps what the source's hooks decided
claude Nov 11, 2026
0e2e479
fix(sdk): GraySwanAnalyzer saves its api_key like every other credential
claude Nov 11, 2026
9bc9055
Merge part 1 (secrets autosave) into part 2
claude Nov 11, 2026
9430e57
test(sdk): the fork's registry is its own, and a replacement agent ke…
claude Nov 11, 2026
1cf42f6
Merge branch 'claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets…
claude Nov 11, 2026
402ecc2
Merge branch 'claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets…
claude Nov 11, 2026
2b1c0b2
refactor(sdk): the fork's copy helper lives with the secret utilities
claude Nov 11, 2026
e076984
Merge branch 'claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets…
claude Nov 11, 2026
f584d5d
Merge branch 'claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets…
claude Nov 11, 2026
8cb7589
Merge branch 'claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets…
claude Nov 13, 2026
4d05f1d
Merge branch 'claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets…
claude Nov 14, 2026
249f4cb
fork: give the fork the hooks the source was created with
claude Nov 14, 2026
cd8f148
Merge branch 'claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets…
claude Nov 14, 2026
115395d
Merge branch 'claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets…
claude Nov 14, 2026
3877994
Merge branch 'claude/openhands-sdk-dual-source-truth-qp1x6r-1-secrets…
claude Nov 20, 2026
bba1de0
fork: keep what a custom secret source holds outside its fields
claude Nov 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -599,7 +599,9 @@ SIDE_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key
both the response and persisted metadata. The title is stored separately
and adds no tag (#4814). Other stored configuration (secrets, plugins,
`parent_conversation_id`, client tools) is copied from the source. The
fork's `last_user_message_id` starts `null`.
fork's `last_user_message_id` is the last user message among the events
it copied, and what a hook decided about a copied action or message
(`blocked_actions`, `blocked_messages`) comes with them.
- The fork's `base_state.json` is written with the source's cipher (the
server's), so the fork's agent keeps its model keys
(`F10.fork-keeps-credentials`) and its conversation secrets; the fork is
Expand Down
126 changes: 126 additions & 0 deletions .agents/skills/verify-sdk/programs/S04/fork_state.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
"""Fork a conversation that was built with a hook and carries a confirmation
policy, a security analyzer with its own model key, and secrets, all set after
construction, and report what the fork has: in memory, in its
`base_state.json`, and after it is loaded again from disk. No model is called.

Usage: fork_state.py
"""

import json
from pathlib import Path

from pydantic import SecretStr
from sdk_verify import emit, make_llm, run_paths

from openhands.sdk import LLM, Agent, LocalConversation
from openhands.sdk.hooks import HookConfig, HookDefinition, HookMatcher
from openhands.sdk.security.confirmation_policy import ConfirmRisky, NeverConfirm
from openhands.sdk.security.risk import SecurityRisk
from openhands.sdk.security.toolshield_llm_analyzer import (
ToolShieldLLMSecurityAnalyzer,
)
from openhands.sdk.utils.cipher import Cipher


paths = run_paths()
BASE = paths["run"] / "fork-state"
WORKSPACE = str(paths["workspace"])
VALUES = ("start-secret-value-1", "guard-key-value-2", "runtime-secret-value-3")
HOOK_REASON = "blocked by the source hook"
cipher = Cipher("sdk-verify-fork-state-key")


def agent() -> Agent:
return Agent(llm=make_llm([{"finish": "unused"}]), tools=[])


def secret_values(conversation) -> dict[str, str | None]:
registry = conversation.state.secret_registry
return {name: registry.get_secret_value(name) for name in registry.secret_sources}


def guard_key(conversation) -> str | None:
analyzer = conversation.state.security_analyzer
if not isinstance(analyzer, ToolShieldLLMSecurityAnalyzer):
return None
key = analyzer.llm.api_key
return key.get_secret_value() if isinstance(key, SecretStr) else None


source = LocalConversation(
agent=agent(),
workspace=WORKSPACE,
persistence_dir=str(BASE),
cipher=cipher,
secrets={"START_TOKEN": VALUES[0]},
hook_config=HookConfig(
user_prompt_submit=[
HookMatcher(
hooks=[HookDefinition(command=f"echo '{HOOK_REASON}' >&2; exit 2")]
)
]
),
visualizer=None,
)
source.set_confirmation_policy(
ConfirmRisky(threshold=SecurityRisk.MEDIUM, confirm_unknown=False)
)
source.set_security_analyzer(
ToolShieldLLMSecurityAnalyzer(
llm=LLM(model="openai/guard", api_key=SecretStr(VALUES[1]), usage_id="guard")
)
)
source.update_secrets({"RUNTIME_TOKEN": VALUES[2]})

fork = source.fork()
fork_dir = Path(fork.state.persistence_dir or "")
saved = json.loads((fork_dir / "base_state.json").read_text())
saved_text = "".join(p.read_text() for p in fork_dir.rglob("*.json"))
out: dict = {
"ok": True,
"fork_policy": fork.state.confirmation_policy.model_dump(mode="json"),
"fork_analyzer": type(fork.state.security_analyzer).__name__,
"fork_analyzer_is_own_object": fork.state.security_analyzer
is not source.state.security_analyzer,
"fork_guard_key_kept": guard_key(fork) == VALUES[1],
"fork_secrets": secret_values(fork),
"saved_policy": saved.get("confirmation_policy"),
"saved_analyzer": (saved.get("security_analyzer") or {}).get("kind"),
"saved_secret_names": sorted(saved["secret_registry"]["secret_sources"]),
"saved_guard_key_is_placeholder": set(
((saved.get("security_analyzer") or {}).get("llm") or {}).get("api_key") or "*"
)
== {"*"},
"plaintext_on_disk": any(value in saved_text for value in VALUES),
}

# A prompt sent to the fork goes through the hook the source was built with.
fork.send_message("hello")
out["fork_prompt_block"] = next(iter(fork.state.blocked_messages.values()), None)

# Changing the fork does not reach back into the source.
fork.set_confirmation_policy(NeverConfirm())
fork.update_secrets({"FORK_ONLY": "fork-only-value-4"})
out["source_policy_after_fork_change"] = type(source.state.confirmation_policy).__name__
out["source_has_fork_only_secret"] = (
"FORK_ONLY" in source.state.secret_registry.secret_sources
)

fork_id = fork.id
fork.close()
source.close()
reloaded = LocalConversation(
agent=agent(),
workspace=WORKSPACE,
persistence_dir=str(BASE),
conversation_id=fork_id,
cipher=cipher,
visualizer=None,
)
out |= {
"reloaded_policy": type(reloaded.state.confirmation_policy).__name__,
"reloaded_guard_key_kept": guard_key(reloaded) == VALUES[1],
"reloaded_secrets": secret_values(reloaded),
}
reloaded.close()
emit(out)
4 changes: 2 additions & 2 deletions .agents/skills/verify-sdk/references/feature-map/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ that map's `Fnn`.

Maintenance baseline: main@141423921fdaeb08cd5f53464a69e860da70b66f (2026-11-19). The next maintenance pass starts from this commit, a pass proposes the next baseline in its PR, and merging that PR accepts it.

The map has **504 sub-features**, **36 of them untested** (no unit or cross
The map has **505 sub-features**, **36 of them untested** (no unit or cross
test exercises the behavior; most of those have an example script).

## What each sub-feature links
Expand Down Expand Up @@ -78,7 +78,7 @@ an interactive login are labeled **blocked**.
| S01 | [Agent step loop](S01-agent-loop.md) | step handling (tool calls, text, empty replies), iteration and budget limits, stuck detector, parallel tools, messages sent mid-run, image input, `ask_agent`, structured output, recovery from bad tool calls and context overflow | `Conversation(max_iteration_per_run=, stuck_detection=, max_budget_per_run=)`, `Agent(tool_concurrency_limit=)`, `ask_agent()`, `response_schema` | llm, tmux, network | 24 | 1 |
| S02 | [LLM core](S02-llm-core.md) | completion, responses and `generate`; streaming; reasoning and thinking blocks; native and prompted tool calling; metrics, stats and pricing; retries and error mapping; caching, token counts, image inlining, completion logs | `LLM(...)`, `llm.completion`/`responses`, `Conversation(token_callbacks=)`, `llm.metrics` | llm, network | 28 | 0 |
| S03 | [LLM registry, routing, fallback, profiles and model switching](S03-llm-routing-profiles.md) | `LLMRegistry`, multimodal and random routers, `FallbackStrategy`, `LLMProfileStore`, `switch_profile`/`switch_llm`, the switch-LLM tool, `route_task_to_model`, subscription login | `LLMRegistry`, `MultimodalRouter`, `FallbackStrategy`, `LLMProfileStore`, `LLM.subscription_login()` | llm, network | 30 | 4 |
| S04 | [Local conversation lifecycle](S04-local-conversation.md) | factory and ids, persistence and resume, callbacks, close, pause, `arun` and interrupt, secrets, visualizers, fork and branches, goal loop (and its stop on a pending `ask_user` question, `GoalAsked`), final response, stats, title and tags | `Conversation(persistence_dir=, callbacks=, visualizer=, secrets=)`, `run`/`pause`/`arun`/`interrupt`/`fork`/`run_goal` | llm, tmux, network, uvx | 34 | 4 |
| S04 | [Local conversation lifecycle](S04-local-conversation.md) | factory and ids, persistence and resume, callbacks, close, pause, `arun` and interrupt, secrets, visualizers, fork and branches, goal loop (and its stop on a pending `ask_user` question, `GoalAsked`), final response, stats, title and tags | `Conversation(persistence_dir=, callbacks=, visualizer=, secrets=)`, `run`/`pause`/`arun`/`interrupt`/`fork`/`run_goal` | llm, tmux, network, uvx | 35 | 4 |
| S05 | [Events and the event log](S05-events.md) | event model, JSON round trip, conversion to LLM messages and OpenAI formats, unmatched actions, the `EventLog`, error classification, completion and token events | `Event.model_validate_json`, `events_to_messages`, `conversation.state.events` | llm, tmux | 18 | 1 |
| S06 | [Agent context, system prompt, condensers and memory](S06-context-condensers.md) | `AgentContext`, system prompt sections and vendor files, persona, persistent memory, the condenser `View`, `LLMSummarizingCondenser`, `condense()`, overflow condensation, no-op and pipeline condensers | `Agent(agent_context=, system_prompt=, condenser=)`, `AgentContext(load_memory=)`, `conversation.condense()` | llm, tmux | 24 | 1 |
| S07 | [Skills](S07-skills.md) | skill sources (programmatic, Markdown, AGENTS.md, SKILL.md, user, project, public, installed), keyword, task and path triggers, `<available_skills>`, `invoke_skill`, inline commands, disabling, install and marketplace, the `OH_PUBLIC_SKILLS_ENABLED` switch | `AgentContext(skills=, load_*_skills=, disabled_skills=)`, `.agents/skills/`, `install_skill`, `OH_PUBLIC_SKILLS_ENABLED` | llm, git, network | 25 | 0 |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ Examples: `examples/01_standalone_sdk/06_interactive_terminal_w_reasoning.py`, `
- `S04.fork`: `fork()` deep-copies the conversation under a new id (or `conversation_id=`), starts IDLE, leaves the source untouched, resets metrics unless `reset_metrics=False`, accepts `agent=` and `tags=` (`title=` is deprecated and ignored locally; only `RemoteConversation` stores it), shares the workspace, and persists so the fork survives a reload. Unit: `tests/sdk/conversation/local/test_fork.py`. Example: `examples/01_standalone_sdk/48_conversation_fork.py`. Real-LLM: none.
- `S04.fork-keeps-secrets`: `fork()` of a `LocalConversation(..., cipher=...)` with a `persistence_dir` creates the fork with the source's cipher and copies the source's conversation secrets (`secrets=`, `update_secrets`), so the fork's `base_state.json` holds the LLM api key and every secret encrypted (not redacted); `LocalConversation(agent=None, conversation_id=fork_id, cipher=...)` reopens the fork with the real key and secrets, and it runs. Without a cipher the fork's key stays redacted on disk. Since [#5765](https://github.com/OpenHands/software-agent-sdk/pull/5765) ([#5465](https://github.com/OpenHands/software-agent-sdk/issues/5465)). Unit: `tests/sdk/conversation/local/test_fork.py::test_fork_keeps_source_cipher_for_persisted_secrets`, `::test_fork_without_cipher_still_redacts_secrets`, `tests/agent_server/test_conversation_service.py::TestConversationTreeForkAndNavigate::test_cipher_backed_fork_resumes_with_credentials`. Example: none. Real-LLM: none.
- `S04.fork-branch`: events carry parent ids forming a tree; `fork(from_event_id=...)` copies only that branch, and `navigate_to(event_id)` moves HEAD so the next event starts a sibling branch (`None` empties the active branch). Unit: `tests/sdk/conversation/local/test_conversation_tree.py`. Example: `examples/01_standalone_sdk/48_conversation_fork.py`. Real-LLM: none.
- `S04.fork-keeps-state`: a fork carries the source's confirmation policy, its own copy of the security analyzer (credentials kept) and the conversation secrets, and is written with the source's cipher, so its `base_state.json` alone restores them; it also runs the hooks in the `hook_config` the source was built with. Unit: `tests/sdk/conversation/local/test_fork.py`, `tests/sdk/conversation/local/test_fork_hook_blocks.py`. Example: none. Real-LLM: none.
- `S04.goal-loop`: `run_goal(conversation, objective, judge_llm, max_iterations=N)` sends the objective, runs, asks the judge LLM for a verdict, re-prompts with the missing items, and returns a `GoalOutcome` with status `"complete"` or `"capped"`, the iteration count and the verdict. Unit: `tests/sdk/conversation/goal/`. Example: `examples/01_standalone_sdk/54_goal_completion_loop.py`. Real-LLM: none.
- `S04.goal-asked`: when a round ends on the agent's `ask_user` question, `run_goal` returns `GoalAsked(ask_id, question)` for it without calling the judge, and the objective it sent is a user message with `sender="goal"`, which does not answer the question; called again while the question is pending it returns the same `GoalAsked` at once, sending nothing and calling neither model; once the user has answered (`conversation.answer_ask(...)` plus the returned message) it runs and the judge can say complete. Unit: `tests/tools/ask_user/test_ask_user.py::test_a_goal_stops_on_a_question_without_judging`. Example: none. Real-LLM: none.
- `S04.goal-asked-branch`: only a question on the active branch stops a goal: after `navigate_to(...)` back to the user message before an unanswered question, `run_goal` with a new objective sends it (`sender="goal"`), runs the agent and asks the judge. Unit: **none (untested)**. Example: none. Real-LLM: none.
Expand Down Expand Up @@ -277,6 +278,22 @@ Scripted bullets use the SDK's `TestLLM` (`--script`, or `make_llm(script=...)`
--check old_branch_in_active eq false --check active_ends_with eq "answer on a new branch" \
--check unknown_event_error contains "Unknown event_id" --check active_after_none eq 0 --check log_after_none eq 7
```
- **What a fork keeps (`S04.fork-keeps-state`).** A ciphered conversation built with a `UserPromptSubmit` hook that blocks every prompt gets `ConfirmRisky(MEDIUM)`, a `ToolShieldLLMSecurityAnalyzer` with its own model key and a runtime secret after construction, then is forked; the fork is sent a message, changed and loaded again from disk.
```sh
export SDK_VERIFY_RUN=$(control-sdk new --field run)
control-sdk program S04/fork_state \
--check fork_policy.kind eq ConfirmRisky --check fork_policy.threshold eq MEDIUM --check fork_policy.confirm_unknown eq false \
--check fork_analyzer eq ToolShieldLLMSecurityAnalyzer --check fork_analyzer_is_own_object eq true --check fork_guard_key_kept eq true \
--check fork_secrets.START_TOKEN eq start-secret-value-1 --check fork_secrets.RUNTIME_TOKEN eq runtime-secret-value-3 \
--check saved_policy.kind eq ConfirmRisky --check saved_analyzer eq ToolShieldLLMSecurityAnalyzer \
--check saved_secret_names.0 eq RUNTIME_TOKEN --check saved_secret_names.1 eq START_TOKEN \
--check saved_guard_key_is_placeholder eq false --check plaintext_on_disk eq false \
--check fork_prompt_block eq "blocked by the source hook" \
--check source_policy_after_fork_change eq ConfirmRisky --check source_has_fork_only_secret eq false \
--check reloaded_policy eq NeverConfirm --check reloaded_guard_key_kept eq true \
--check reloaded_secrets.START_TOKEN eq start-secret-value-1 --check reloaded_secrets.FORK_ONLY eq fork-only-value-4
```
The fork has the policy with its threshold, an analyzer object of its own whose key still works, and both secrets; its `base_state.json` holds them encrypted (neither the plaintext nor the `**********` placeholder). The message sent to the fork is blocked with the source hook's reason. Setting `NeverConfirm` and a new secret on the fork leaves the source as it was, and the fork loaded again from disk has the fork's values.
- **The goal loop (`S04.goal-loop`).** `run_goal` with a scripted judge: incomplete then complete (the follow-up names what was missing), and a capped run whose last verdict cannot be parsed.
```sh
export SDK_VERIFY_RUN=$(control-sdk new --field run)
Expand Down
Loading
Loading