Skip to content

fix(aws-strands): preserve structured system prompts per thread - #3386

Open
contextablemark wants to merge 3 commits into
mainfrom
albert-issue-3384
Open

contextablemark wants to merge 3 commits into
mainfrom
albert-issue-3384

Conversation

@contextablemark

@contextablemark contextablemark commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Python per-thread Strands agents currently reconstruct the system prompt from its text-only compatibility view, dropping cache checkpoints and separate content blocks. Capture the canonical structured prompt as a value snapshot and copy it independently for each thread, retaining the text fallback for Agent-like templates.

TypeScript already preserves structured blocks, but shares their mutable contents across threads. Copy those values while preserving SDK block classes, guard content, and image bytes. Keep each adapter's existing per-thread override policy and cached-agent reuse.

Reject unrecognized SDK system block kinds with a descriptive TypeError instead of stripping their prototypes. An exhaustive type check flags newly added SDK block kinds at build time; plain prompt data continues to be copied normally.

Fixes #3384.

Validation

  • Regression tests failed before the implementation and pass afterward in both languages.
  • Python related suites: 250 passed / 9 skipped on SDK 1.55.0; 260 passed / 5 skipped on 1.64.0. The full locked, minimum, and latest Python CI lanes also passed for the implementation commit.
  • Full TypeScript suites, with Nx cache disabled: 1,881 passed / 53 skipped on SDK 1.1.0; 1,935 passed / 1 skipped on SDK 1.24.0.
  • TypeScript typecheck and build passed on both versions. Python sdist/wheel build, formatting, and scoped lint checks passed.
  • Tests cover model-bound prompt parity with native SDK calls, canonical stored prompt content, checkpoint TTLs, template snapshots, independent raw image bytes, thread isolation, compatibility inputs, and existing override/reuse behavior.

Validation limits

SDK 1.24.0's native model middleware double-encodes guard-image bytes even without this adapter. Model-bound tests compare against a separately invoked native Agent, while independent assertions require unchanged raw bytes and isolated buffers in the adapter's stored prompts. This PR does not patch that SDK behavior.

The TypeScript build exits successfully with declaration-bundler warnings. No live provider cache-hit or billing measurements were performed.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Python Preview Packages

Version 0.0.0.dev1794592803 published to TestPyPI.

Warning: These packages are built from contributor code that may not yet have been vetted for correctness or security. Install at your own risk and do not use in production.

Install with uv

Add the TestPyPI index to your pyproject.toml:

[[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
explicit = true

Then install the packages you need:

# Core SDK
uv add 'ag-ui-protocol==0.0.0.dev1794592803' --index testpypi

# Integrations (each already depends on the matching ag-ui-protocol preview)
uv add 'ag-ui-langgraph==0.0.0.dev1794592803' --index testpypi
uv add 'ag-ui-crewai==0.0.0.dev1794592803' --index testpypi
# NOTE: ag-ui-agent-spec depends on pyagentspec (git-only, not on PyPI).
# You will need to install pyagentspec separately from its git repo.
uv add 'ag-ui-agent-spec==0.0.0.dev1794592803' --index testpypi
uv add 'ag_ui_adk==0.0.0.dev1794592803' --index testpypi
uv add 'ag_ui_strands==0.0.0.dev1794592803' --index testpypi

Install with pip

pip install \
  --index-url https://test.pypi.org/simple/ \
  --extra-index-url https://pypi.org/simple/ \
  ag-ui-protocol==0.0.0.dev1794592803

Use --extra-index-url https://pypi.org/simple/ so pip can resolve
transitive dependencies (pydantic, fastapi, etc.) from real PyPI.


Commit: e22d472

@pkg-pr-new

pkg-pr-new Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

@ag-ui/a2a-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2a-middleware@3386

@ag-ui/a2ui-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2ui-middleware@3386

@ag-ui/event-throttle-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/event-throttle-middleware@3386

@ag-ui/mcp-apps-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/mcp-apps-middleware@3386

@ag-ui/mcp-middleware

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/mcp-middleware@3386

@ag-ui/a2a

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2a@3386

@ag-ui/adk-js

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/adk-js@3386

@ag-ui/adk

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/adk@3386

@ag-ui/ag2

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/ag2@3386

@ag-ui/agno

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/agno@3386

@ag-ui/antigravity

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/antigravity@3386

@ag-ui/aws-strands

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/aws-strands@3386

@ag-ui/claude-agent-sdk

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/claude-agent-sdk@3386

@ag-ui/claude-managed-agents

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/claude-managed-agents@3386

@ag-ui/crewai

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/crewai@3386

@ag-ui/langchain

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/langchain@3386

@ag-ui/langgraph

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/langgraph@3386

@ag-ui/llamaindex

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/llamaindex@3386

@ag-ui/mastra

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/mastra@3386

@ag-ui/openai-agents

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/openai-agents@3386

@ag-ui/pydantic-ai

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/pydantic-ai@3386

@ag-ui/vercel-ai-sdk

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/vercel-ai-sdk@3386

@ag-ui/watsonx

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/watsonx@3386

@ag-ui/a2ui-toolkit

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/a2ui-toolkit@3386

create-ag-ui-app

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/create-ag-ui-app@3386

@ag-ui/client

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/client@3386

@ag-ui/core

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/core@3386

@ag-ui/encoder

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/encoder@3386

@ag-ui/proto

pnpm add https://pkg.pr.new/ag-ui-protocol/ag-ui/@ag-ui/proto@3386

commit: d130ccb

@contextablemark
contextablemark marked this pull request as ready for review November 12, 2026 17:53
@contextablemark
contextablemark requested a review from a team as a code owner November 12, 2026 17:53
@contextablemark

Copy link
Copy Markdown
Contributor Author

@NathanTarbert Could you take a look?

@surlymochan surlymochan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the Python side locally against this branch (HEAD e2d7787): pytest tests/test_system_prompt_preservation.py gives 13 passed. As a control I reverted the two agent.py hunks (wrap-time snapshot + per-thread deepcopy) with the new tests in place: 9 failed, 4 passed, so the tests genuinely pin the fix (failing cases include the cache-point TTL survival and the reserved-override test).

Two things I checked while reading:

  1. The wrap-time snapshot uses getattr(agent, "system_prompt_content", agent.system_prompt). The legacy-agent test covers the fallback path, and the snapshot happens before any thread is created, so a later template.system_prompt = ... reassignment cannot leak in (covered by test_prompt_is_a_value_snapshot_at_wrap_time).
  2. The per-thread deepcopy(self._system_prompt) in _run_raw means a thread mutating its own blocks cannot poison the template or sibling threads (covered by test_threads_do_not_share_prompt_blocks_and_cached_threads_are_reused).

I did not run the TypeScript suite (no installed node_modules for that package in my checkout), but the TS mirror applies the same snapshot-at-wrap + copy-per-thread shape, and its tests assert the same invariants including SDK block prototype survival. No blocking issue from me.

@surlymochan surlymochan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the Python and TypeScript diffs plus the new tests.

Verified:

  • Python: snapshotting system_prompt_content (with the text fallback for Agent-like templates) at wrap time, then deepcopy per thread, matches the stated fix. The new tests pin the three properties that matter: canonical blocks reach the model, threads do not share blocks, and cached threads are still reused.
  • TypeScript: _copySystemPrompt rebuilds the three handled block kinds through their SDK classes, and the tests prove mutation isolation including guard image bytes (bytes[0] = 99 on the copy leaves the original alone).

One question:

  • The _copySystemPrompt switch handles textBlock, cachePointBlock, and guardContentBlock; any other block falls through to structuredClone(block), which returns a plain object and drops the SDK prototype. The PR description says SDK block classes are preserved. Can a top-level image block (or any other SDK block kind) appear in systemPrompt, or are images only ever nested inside guard content (as the test covers)? If another block kind is reachable at the top level, it would silently lose its class methods after copying.

@surlymochan surlymochan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against the strands SDK source (strands-agents 1.65.0) as well as the diff and the new tests:

Verified:

  • The fix snapshots the right accessor. In the SDK, Agent.system_prompt returns the compat string built by split_system_prompt(...)[0], which drops every non-text key (the SDK itself warns about dropped keys at agent.py:1172), so the old code lost cache points. Agent.system_prompt_content returns the canonical blocks but only as a shallow outer-list copy (agent.py:838), so nested dicts are still shared — the deepcopy at wrap time is the correct remedy, and it matches what the SDK itself does when serializing state (copy.deepcopy(self._system_prompt_content), agent.py:2281).
  • deepcopy is imported at agent.py:15, so the new call sites cannot NameError.
  • The per-thread deepcopy(self._system_prompt) at thread creation is load-bearing, not belt-and-braces: the thread-isolation test mutates thread A's nested text and cachePoint ttl plus appends a block, then proves thread B and the template still equal BLOCKS while cached thread A is reused with its own mutations.
  • The wrap-time snapshot test proves the fix: after wrapping, the source list is mutated, a block appended, and template.system_prompt replaced with a string — the model still receives the original BLOCKS.
  • The legacy fallback works: a template without system_prompt_content (SimpleNamespace) goes through getattr(..., agent.system_prompt) and the model receives [[{"text": "Legacy prompt"}]], degrading gracefully instead of crashing.
  • The provider-level proof is real: BedrockModel.format_request with the recorded prompts yields request["system"] == BLOCKS, so the cache point and ttl survive to the actual provider request, with dummy credentials and no network call.
  • The thread.system_prompt == template.system_prompt assertion is sound, not a type mix-up: both sides are strands Agents and the property returns the compat string derived from the same blocks, while model.prompts == [expected] separately pins that the model receives the structured blocks.

One question: the SDK version this pins against — system_prompt_content exists in 1.65.0, but if the supported strands range includes a version without that property, the fallback silently loses cache points again (no crash, just the old behavior). Is the minimum strands version already new enough that the fallback is dead code, or is it load-bearing for older installs? If the latter, a comment naming the version floor would help future readers.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: aws-strands (Python): per-thread agents get the template's system prompt as one string, dropping cache points and content blocks

2 participants