Repository navigation
fix(sdk): a fork keeps the source's confirmation policy, security analyzer and hooks - #6763
Conversation
LocalConversation.fork() built the fork without the source's cipher, so the fork's base_state.json stored the LLM key redacted. The agent-server resumes every fork from that file, so the fork's first turn failed with LLMAuthenticationError. Forward the cipher, and copy the source's conversation secrets into the fork's registry so they persist too. The verify-agent-server map bullets F10.fork-keeps-credentials and F10.fork-run-memory now pass; their known-bug markers are removed. Fixes #5465 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014MKndVj7XWvRQmvW9w2bDg
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
must carry, and it does the registry copy better than #4813 did: a fresh SecretRegistry with deep-copied sources, so the fork does not share the source's masking cache or its lock. Merging it here instead of re-implementing it keeps one version of that code, with its authors and its tests (kyletser's restart test included), whichever PR lands first. Conflict: the feature-map index count. main's 955 stands (`map check` agrees); #5765 removes two known-bug markers and adds no sub-feature. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…ersation.fork() These fields are part of ConversationState but fork() did not copy them, so the fork's base_state.json was written with the NeverConfirm / None defaults and the source conversation's policy was silently lost: a fork of a conversation under AlwaysConfirm ran its actions unasked. Split out of #4813 (commit 61b865e). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
… kept The fork shared the source's analyzer object, so a stateful analyzer's history (e.g. ToolShieldLLMSecurityAnalyzer) mixed two conversations. Give the fork its own instance through a JSON round-trip with expose_secrets, so the fork's cipher re-encrypts nested credentials (the guardrail LLM api_key) instead of writing them redacted. Tests: the fork owns its analyzer, a ciphered analyzer key survives a resume of the fork, and secrets passed at start reach the fork (the registry copy itself comes from #5765). Split out of #4813 (commit f6ade80). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
The policy copy had no test. This one checks the fork's live state and its base_state.json, which is what the agent-server resumes a fork from. It fails on main, where the fork starts with NeverConfirm. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
New sub-feature S04.fork-keeps-state with a program that forks a ciphered conversation carrying ConfirmRisky(MEDIUM), a ToolShield analyzer with its own key and a runtime secret, then changes the fork and loads it again from disk. The recipe checks the fork's memory, its base_state.json and the reload. On main fifteen of its checks fail: the fork starts with NeverConfirm, no analyzer and no secrets, and its saved key is the placeholder. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
Review of the fork changes found three small things:
- Secret sources were deep-copied. A custom source that holds a lock or
a client made fork() raise ("cannot pickle '_thread.lock' object"),
after the fork's directory had already been written. They are now
copied like the agent and the analyzer, through a dump/validate round
trip (`_own_copy`), and the analyzer and the secrets are copied before
the fork is constructed, so a copy that fails leaves nothing behind.
- `invoked_skills` was not copied, next to two skill lists that are.
- The analyzer test passed with a shallow copy. It now uses a
ToolShield analyzer and checks that the guardrail LLM is the fork's
own object too.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
Coverage Report •
|
||||||||||||||||||||||||||||||
fork() copied the events but not the hook verdicts recorded against them. A pending action that a PreToolUse hook had denied was executed when the fork ran, and a user message that a UserPromptSubmit hook had blocked was sent to the model, while the source did neither. This is on main too; it is the same gap as the confirmation policy, a safety decision that a fork silently dropped, so it is closed here. The fork takes over `blocked_actions` and `blocked_messages` for the events it copies, and `last_user_message_id` (the last user message it actually has, for a fork of an earlier branch point). Found in review of this stack. Each new test fails without the change: the denied command runs, the blocked prompt reaches the model. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
`api_key` was a bare SecretStr with no serializer, so a saved conversation state always held the redaction placeholder for it, even with a cipher, and loading turned that placeholder into the key: the analyzer sent `Authorization: Bearer **********`. Only the GRAYSWAN_API_KEY environment variable hid this. The field now goes through `serialize_secret` / `validate_secret`: with a cipher the key is encrypted and comes back; without one it loads as no key, and the analyzer reports UNKNOWN as it does when none is set. This matters for this stack because a fork, and from the last PR every reload, takes the analyzer from base_state.json alone. Found in review of this stack. Both new tests fail without the change. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
Brings in the review fixes made on part 1: the shorter ACP masker comment, the masked string form of state update events with its test, and the feature-map note about changes made during a step. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…eps its secrets Two properties of the registry copy that had no test: - Replacing a secret on the fork must not change what the source masks. With a registry copied by `model_copy`, as #4813 first did, the two share one masking cache and the source stops masking its own value once the fork has resolved the new one. The test fails on that copy. - `fork(agent=...)` with an agent that brings secrets in its context has them next to the ones copied from the source. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
The helper that copies a model through its own dump and validate, with credentials kept, moves from local_conversation.py to utils/pydantic_secrets.py as copy_keeping_secrets. The agent-server makes the same kind of copy for a child conversation and can import it instead of spelling the round trip out again. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state # Conflicts: # openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
|
Heads-up for the stack: the #5465 part of this PR (cipher + secret registry on fork) comes from #5765, which is being driven to merge on its own now (merged with current This comment/PR was created by an AI agent (OpenHands) on behalf of Graham Neubig. |
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
Part of the fork's state was the hooks' verdicts on the events it copies, but not the hooks. A pending action the source's PreToolUse hook had denied stayed denied in the fork, and the next action the fork asked for ran with no hook at all. The fork is now built with the caller's own hook_config. The plugin hooks are not passed along: the fork loads the plugins itself, and the merged config would register them twice. Hooks of a plugin the source loaded at runtime are still not carried. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state # Conflicts: # .agents/skills/verify-agent-server/references/feature-map/F10-fork-navigate-condense-ask.md # .agents/skills/verify-sdk/references/feature-map/README.md
…-autosave' into claude/openhands-sdk-dual-source-truth-qp1x6r-2-fork-keeps-state main at 3d638f2 brings #5765 (the fork's cipher and secret registry) and the SecretField refactor. What part 2 still adds to fork() stays on top of main's version: the policy, the analyzer copy, the hooks and their verdicts, invoked_skills, and one copy helper for the agent, the analyzer and the secret sources. The helper builds its context with client_context, and GraySwanAnalyzer.api_key is declared with SecretField instead of the deprecated validator pair. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
Agent-server conversation start benchmarkBase
How this is measuredEach create is a Canvas-shaped
A timing is flagged when the 95% CIs do not overlap and the medians differ by at least 50 ms, or when a row with at least 8 samples per side moves by at least 150 ms and 25%. A create stalled the server when its slowest |
The fork copied every secret source through a dump and validate round trip. A custom `SecretSource` that keeps its value in a private attribute or in a field the dump leaves out came out of that with no value (and a required field of that kind made `fork()` raise), where main's deep copy (#5765) keeps it. The same held for such a source in the agent's context. Secret sources are deep-copied again, as on main. A source the deep copy cannot handle (one that holds a lock or an open client, which makes `fork()` raise on main) is rebuilt from its fields instead. Tests for each case; each fails with only the other half in place. Found by the review of the merge with main. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY
HUMAN:
Engel Nyst asked me to find the fork PR that removes the two sources of truth for a conversation's settings (
meta.jsonin the agent-server andbase_state.jsonin the SDK), which is #4813 by @BSmick6; to put its commits on branches in this repo, split so that each concern can be reviewed on its own; and to open new PRs that credit the author and say that we do the final review and tweaks. Then to review each PR, address what the review finds, and prove that the changes work by running the SDK and the agent-server the way a program or an agent does, with the repo'sverify-sdkandverify-agent-serverskills. The principle behind the request: the agent-server is a REST interface to the SDK. It should not be a different thing, and it should not repeat what the SDK already does, unless a particular case gives a special reason.This is part 2 of 3: a fork keeps the source's confirmation policy, security analyzer, hooks and hook verdicts.
AGENT:
I'm Claude (Fable 5.2), an AI agent working for Engel Nyst. I brought #4813 up to date with
main, split it into a stack of three PRs, reviewed each, and verified them against real servers. The policy and analyzer copy in this PR is @BSmick6's, from #4813, with the analyzer handling from Engel Nyst's follow-up on that PR. #5765, which gives a fork its source's cipher and secrets, merged intomainwhile this PR was open, so that part is no longer in this diff. What the final review added is listed under "Final review".The stack: 1. #6762 (secrets are saved the moment they change) → 2. this PR → 3. #6764 (the agent-server stops keeping its own copy). This PR is based on part 1, so its diff shows only its own change.
Why
LocalConversation.fork()builds the fork from the agent and the events and, since #5765, gives it the source's cipher and a copy of its secrets. It does not copy the confirmation policy or the security analyzer. Onmain:AlwaysConfirmorConfirmRiskystarts withNeverConfirmand runs its actions unasked;The agent-server hides part of this: it copies the source's
meta.jsonto the fork and applies it on every load, so a server fork gets the policy and analyzer its source was created with, not ones set later (see Notes). #6764 removes that second copy, so the SDK has to carry these values itself.Summary
confirmation_policy(@BSmick6, fix(agent-server): own confirmation_policy, security_analyzer, secrets in ConversationState #4813).security_analyzerthrough a round trip that keeps nested credentials, so a stateful analyzer does not mix two conversations' history and a guardrail model key is re-encrypted for the fork instead of written redacted (Engel Nyst's follow-up on fix(agent-server): own confirmation_policy, security_analyzer, secrets in ConversationState #4813).main. This branch adds one thing there: a secret source that holds a lock or an open client no longer makesfork()fail; it is rebuilt from its fields (item 3 below).Final review: what was found and changed
blocked_actions,blocked_messages,last_user_message_id) and were not copied. I reproduced it onmain: a pending action that aPreToolUsehook had denied on the source was executed when the fork ran, and a prompt that aUserPromptSubmithook had blocked was sent to the model. The fork now carries the verdicts for the events it copies, and the hooks themselves: it is built with thehook_configits source was created with. The verdicts alone would have been half a fix, since the next action the model asked for in an SDK-level fork ran without the caller'shook_confighooks; only the hooks of plugins given at construction, which the fork loads itself, still ran (a fork made through the agent-server kept explicit hooks, because the server passes its storedhook_configwhen it loads the fork). The plugin hooks are not handed over: the fork loads the plugins itself, and the merged config would register them twice. Six tests, all failing without the change.GraySwanAnalyzernever kept itsapi_keyacross a save, cipher or not: the field had no serializer, so it was written as the placeholder and read back as the literal string**********, which the analyzer then sent as its bearer token. This showed only when the key was passed as a field andGRAYSWAN_API_KEYwas not set, because that variable always replaces the field. The field is now declared withSecretField, like the SDK's other credentials since refactor(sdk): declare secret fields once with SecretField #6779: with a cipher the key is encrypted and comes back; without one it is still written as the placeholder but loads as no key, and the analyzer reportsUNKNOWNrisk instead of sending the placeholder. Two tests, both failing without the change. Found while checking what a forked analyzer has after a reload.copy_keeping_secrets, next to the SDK's other secret utilities, so the agent-server can use the same one for child conversations). Onmainthat round trip is spelled out infork()for the agent; for the analyzer it is also what keeps one conversation's history out of the other's. The secret sources are deep-copied, as onmainsince fix(sdk): keep the source's cipher and secrets on a forked conversation #5765, because a custom source may keep its value outside its fields (in a private attribute), where a dump does not carry it.model_copy(deep=True)alone raisesTypeErroron a custom source that holds a lock (cannot pickle '_thread.lock' object) or an HTTP client, sofork()fails onmainfor such a source; here it is rebuilt from its fields instead. That rebuild is the same dump and validate, with its limit: a source that both cannot be deep-copied and keeps its value outside its fields comes out without the value (the fork resolves it to nothing, with no error), andfork()still raises if the fields alone do not validate. The agent's own copy is a round trip, as onmain, so such a source inagent_context.secretshas no value in the fork's agent; the fork's registry has the working copy. The built-in sources,StaticSecretandLookupSecret, are not affected either way. The analyzer and the secret sources are copied under the source's lock, before the fork object exists, so a copy that fails leaves no fork directory behind. The agent's copy stays where it was onmain, before the lock is taken. An earlier version of this PR sent the secret sources through the round trip as well; the review after the last merge ofmaincaught that it dropped a privately held value, and there is a test for each case now.invoked_skillsis copied too, next to the skill listsfork()already copied, so the fork reports the skills its source invoked (conversation.state.invoked_skills, which is also what the agent-server's conversation info shows asinvoked_skills) instead of an empty list.fork(agent=...)keeps its secrets. The last two pass onmainas well since fix(sdk): keep the source's cipher and secrets on a forked conversation #5765; they stay as regression tests. Newverify-sdksub-featureS04.fork-keeps-state, with a program that forks a ciphered conversation, changes the fork and loads it again from disk.Issue Number
Part of #4810.
How to Test
With this branch's tests on the code it is based on (
mainplus #6762), 13 of the command's 226 tests fail: five intest_fork.py(test_fork_keeps_confirmation_policy,test_fork_gives_independent_security_analyzer,test_fork_preserves_ciphered_security_analyzer_credentials,test_fork_copies_a_secret_source_that_cannot_be_deep_copied,test_fork_copies_invoked_skills), all six intest_fork_hook_blocks.py(two withassert ['drop-the-data'] == []: the command the hook denies ran), and the two GraySwan tests. On this branch all 226 pass.In this section
mainismainat3d638f2e, the commit this branch is merged with. Live, with the repo'sverify-sdkskill. The recipe forks a ciphered conversation that hasConfirmRisky, an analyzer with its own model key, two secrets and a hook that blocks every prompt; it then changes the fork and loads it again from disk.maindoes not have this bullet, so its column is this branch's recipe and program placed in a copy ofmainand run onmain's SDK.mainS04.fork-keeps-stateThe checks that fail on
main, with what the program got instead:fork_policy.kind eq ConfirmRisky: gotNeverConfirmfork_policy.threshold eq MEDIUM: gotnullfork_policy.confirm_unknown eq false: gotnullfork_analyzer eq ToolShieldLLMSecurityAnalyzer: gotNoneTypefork_guard_key_kept eq true: gotfalsesaved_policy.kind eq ConfirmRisky: gotNeverConfirmsaved_analyzer eq ToolShieldLLMSecurityAnalyzer: gotnullsaved_guard_key_is_placeholder eq false: gottruefork_prompt_block eq blocked by the source hook: gotnullreloaded_guard_key_kept eq true: gotfalsereloaded_secrets.FORK_ONLY eq fork-only-value-4: gotnullThree of these follow from the missing analyzer rather than from a key that was written badly:
mainsaves no analyzer for the fork, so there is no guard key at all, and the program reports a missing key as a placeholder (fork_guard_key_kept,saved_guard_key_is_placeholder,reloaded_guard_key_kept).On the code this branch is based on (
mainplus #6762) the same recipe fails 10 of these, twice. The one that passes there isreloaded_secrets.FORK_ONLY eq fork-only-value-4: a secret added to the fork is still there after a reload, which is #6762's fix, not this one's.The neighbouring bullets (
S04.fork,S04.fork-keeps-secrets,S04.fork-branch) pass onmainand on this branch, in the whole-family runs below;S04.fork-keeps-secretsis #5765's. The two agent-server bullets this PR fixed before #5765 merged (F10.fork-keeps-credentials,F10.fork-run-memory) pass onmainnow, and on the top of the stack (#6764's table).On this branch's final commit (
bba1de01), the two SDK map families:S04local conversation (SDK map)main(S04.pause-during-startup,S04.goal-asked-branch,S04.interactive-terminal);S04.factorylabelled blocked in the mapS13security and confirmation (SDK map)S13.confirm-approve-reject,S13.llm-analyzerlabelled blocked in the mapS04.interactive-terminalends with an integration test whose command asks for the modeldeepseek/deepseek-flashby name. This run has an LLM proxy and no DeepSeek key, so the request goes to the proxy, which serves the flash model under another name and answers 400 (There are no healthy deployments for this model). The example step before it passes, and the bullet fails the same way onmain.S04.pause-during-startupandS04.goal-asked-branchare bulletsmain's map maintenance added this week (#6869, #6931); they fail the same checks onmain(compared check by check) and concern pausing and goals, which this change does not touch.The whole agent-server families (
F04,F08,F10,F12,F31) and the client entry points were replayed on the top of the stack, which contains this commit; that table is in #6764.GitHub CI on
bba1de01: all 15 checks that ran are green, and no job needed a re-run. Two are skipped by design: the macOS matrix placeholder, and the description check, which waits for the PR to leave draft.Type
Notes
F10map edits. What stays is the policy, the analyzer, the hook verdicts and hooks,invoked_skills, the copy helper (with fix(sdk): keep the source's cipher and secrets on a forked conversation #5765's deep copy of the secret sources kept, plus a fallback for a source that cannot be deep-copied), the GraySwan fix, theS04.fork-keeps-staterecipe and their tests.meta.jsonwhen it opens the fork: the policy and analyzer are the ones the source was created with, and a secret from the start request that was changed later has its original value again. A secret added after creation is kept, with its value only if the server has a cipher. fix(agent-server): one owner for confirmation_policy, security_analyzer and secrets (base_state.json) #6764 has the server-level recipes for that (F10.fork-keeps-settings,F08.fork-after-restart).SessionStarton first use andSessionEndon close included. A fork made through the agent-server is not affected: the server never makes the SDK fork object ready, and loads the fork from disk with its stored hooks, as before.load_plugin(), or custom stuck-detection thresholds.mainand left for it:fork(reset_metrics=False)of a conversation that was resumed from disk stops counting what the fork spends, so the fork's budget cap never trips. Only an SDK fork used in-process is affected; a server fork is reloaded from disk and counts. I have a reproduction (a test file that fails onmain) and a suggested fix for an issue.maindo not run on it (among them REST contract, persisted settings, OpenAPI schema, binary and image build, TypeScript client). fix(agent-server): one owner for confirmation_policy, security_analyzer and secrets (base_state.json) #6764 lists local runs of the OpenAPI, REST-contract, persisted-settings and SDK-API checks on the top of the stack; the binary build, the image build and the TypeScript client CI were not run locally (four conversation bullets of the TypeScript client's map familyF34were run live on the top of the stack: create, run, fork/navigate/ask, confirm; they are in fix(agent-server): one owner for confirmation_policy, security_analyzer and secrets (base_state.json) #6764's table).🤖 Generated with Claude Code
https://claude.ai/code/session_013ArwwMuszm1Nt9r2whddpY