Repository navigation
fix(aws-strands): preserve structured system prompts per thread - #3386
contextablemark wants to merge 3 commits into
Conversation
Python Preview PackagesVersion
Install with uvAdd the TestPyPI index to your [[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
explicit = trueThen install the packages you need: # Core SDK
uv add 'ag-ui-protocol==0.0.0.dev1794592803' --index testpypi
# Integrations (each already depends on the matching ag-ui-protocol preview)
uv add 'ag-ui-langgraph==0.0.0.dev1794592803' --index testpypi
uv add 'ag-ui-crewai==0.0.0.dev1794592803' --index testpypi
# NOTE: ag-ui-agent-spec depends on pyagentspec (git-only, not on PyPI).
# You will need to install pyagentspec separately from its git repo.
uv add 'ag-ui-agent-spec==0.0.0.dev1794592803' --index testpypi
uv add 'ag_ui_adk==0.0.0.dev1794592803' --index testpypi
uv add 'ag_ui_strands==0.0.0.dev1794592803' --index testpypiInstall with pippip install \
--index-url https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
ag-ui-protocol==0.0.0.dev1794592803
Commit: e22d472 |
@ag-ui/a2a-middleware
@ag-ui/a2ui-middleware
@ag-ui/event-throttle-middleware
@ag-ui/mcp-apps-middleware
@ag-ui/mcp-middleware
@ag-ui/a2a
@ag-ui/adk-js
@ag-ui/adk
@ag-ui/ag2
@ag-ui/agno
@ag-ui/antigravity
@ag-ui/aws-strands
@ag-ui/claude-agent-sdk
@ag-ui/claude-managed-agents
@ag-ui/crewai
@ag-ui/langchain
@ag-ui/langgraph
@ag-ui/llamaindex
@ag-ui/mastra
@ag-ui/openai-agents
@ag-ui/pydantic-ai
@ag-ui/vercel-ai-sdk
@ag-ui/watsonx
@ag-ui/a2ui-toolkit
create-ag-ui-app
@ag-ui/client
@ag-ui/core
@ag-ui/encoder
@ag-ui/proto
commit: |
|
@NathanTarbert Could you take a look? |
surlymochan
left a comment
There was a problem hiding this comment.
Verified the Python side locally against this branch (HEAD e2d7787): pytest tests/test_system_prompt_preservation.py gives 13 passed. As a control I reverted the two agent.py hunks (wrap-time snapshot + per-thread deepcopy) with the new tests in place: 9 failed, 4 passed, so the tests genuinely pin the fix (failing cases include the cache-point TTL survival and the reserved-override test).
Two things I checked while reading:
- The wrap-time snapshot uses
getattr(agent, "system_prompt_content", agent.system_prompt). The legacy-agent test covers the fallback path, and the snapshot happens before any thread is created, so a latertemplate.system_prompt = ...reassignment cannot leak in (covered bytest_prompt_is_a_value_snapshot_at_wrap_time). - The per-thread
deepcopy(self._system_prompt)in_run_rawmeans a thread mutating its own blocks cannot poison the template or sibling threads (covered bytest_threads_do_not_share_prompt_blocks_and_cached_threads_are_reused).
I did not run the TypeScript suite (no installed node_modules for that package in my checkout), but the TS mirror applies the same snapshot-at-wrap + copy-per-thread shape, and its tests assert the same invariants including SDK block prototype survival. No blocking issue from me.
surlymochan
left a comment
There was a problem hiding this comment.
Reviewed the Python and TypeScript diffs plus the new tests.
Verified:
- Python: snapshotting
system_prompt_content(with the text fallback for Agent-like templates) at wrap time, thendeepcopyper thread, matches the stated fix. The new tests pin the three properties that matter: canonical blocks reach the model, threads do not share blocks, and cached threads are still reused. - TypeScript:
_copySystemPromptrebuilds the three handled block kinds through their SDK classes, and the tests prove mutation isolation including guard image bytes (bytes[0] = 99on the copy leaves the original alone).
One question:
- The
_copySystemPromptswitch handlestextBlock,cachePointBlock, andguardContentBlock; any other block falls through tostructuredClone(block), which returns a plain object and drops the SDK prototype. The PR description says SDK block classes are preserved. Can a top-level image block (or any other SDK block kind) appear insystemPrompt, or are images only ever nested inside guard content (as the test covers)? If another block kind is reachable at the top level, it would silently lose its class methods after copying.
surlymochan
left a comment
There was a problem hiding this comment.
Verified against the strands SDK source (strands-agents 1.65.0) as well as the diff and the new tests:
Verified:
- The fix snapshots the right accessor. In the SDK,
Agent.system_promptreturns the compat string built bysplit_system_prompt(...)[0], which drops every non-textkey (the SDK itself warns about dropped keys at agent.py:1172), so the old code lost cache points.Agent.system_prompt_contentreturns the canonical blocks but only as a shallow outer-list copy (agent.py:838), so nested dicts are still shared — thedeepcopyat wrap time is the correct remedy, and it matches what the SDK itself does when serializing state (copy.deepcopy(self._system_prompt_content), agent.py:2281). deepcopyis imported at agent.py:15, so the new call sites cannot NameError.- The per-thread
deepcopy(self._system_prompt)at thread creation is load-bearing, not belt-and-braces: the thread-isolation test mutates thread A's nested text and cachePoint ttl plus appends a block, then proves thread B and the template still equal BLOCKS while cached thread A is reused with its own mutations. - The wrap-time snapshot test proves the fix: after wrapping, the source list is mutated, a block appended, and
template.system_promptreplaced with a string — the model still receives the original BLOCKS. - The legacy fallback works: a template without
system_prompt_content(SimpleNamespace) goes throughgetattr(..., agent.system_prompt)and the model receives[[{"text": "Legacy prompt"}]], degrading gracefully instead of crashing. - The provider-level proof is real:
BedrockModel.format_requestwith the recorded prompts yieldsrequest["system"] == BLOCKS, so the cache point and ttl survive to the actual provider request, with dummy credentials and no network call. - The
thread.system_prompt == template.system_promptassertion is sound, not a type mix-up: both sides are strands Agents and the property returns the compat string derived from the same blocks, whilemodel.prompts == [expected]separately pins that the model receives the structured blocks.
One question: the SDK version this pins against — system_prompt_content exists in 1.65.0, but if the supported strands range includes a version without that property, the fallback silently loses cache points again (no crash, just the old behavior). Is the minimum strands version already new enough that the fallback is dead code, or is it load-bearing for older installs? If the latter, a comment naming the version floor would help future readers.
Python per-thread Strands agents currently reconstruct the system prompt from its text-only compatibility view, dropping cache checkpoints and separate content blocks. Capture the canonical structured prompt as a value snapshot and copy it independently for each thread, retaining the text fallback for Agent-like templates.
TypeScript already preserves structured blocks, but shares their mutable contents across threads. Copy those values while preserving SDK block classes, guard content, and image bytes. Keep each adapter's existing per-thread override policy and cached-agent reuse.
Reject unrecognized SDK system block kinds with a descriptive
TypeErrorinstead of stripping their prototypes. An exhaustive type check flags newly added SDK block kinds at build time; plain prompt data continues to be copied normally.Fixes #3384.
Validation
Validation limits
SDK 1.24.0's native model middleware double-encodes guard-image bytes even without this adapter. Model-bound tests compare against a separately invoked native Agent, while independent assertions require unchanged raw bytes and isolated buffers in the adapter's stored prompts. This PR does not patch that SDK behavior.
The TypeScript build exits successfully with declaration-bundler warnings. No live provider cache-hit or billing measurements were performed.