Python: Preserve MCP Host payloads in AG-UI history

[Eduard van Valkenburg (eavanvalkenburg)]

Motivation & Context

MCP Apps need model-visible tool output and Host/UI hydration data to remain separate. Agent Framework currently loses the complete MCP CallToolResult when AG-UI replaces streamed events with MESSAGES_SNAPSHOT or rebuilds persisted history, causing widgets to disappear after a run or reload.

This is orthogonal to #7866 and draft PR #7897. That work selects and deduplicates model-visible MCP content; this change preserves the complete Host payload across AG-UI surfaces without exposing UI-only JSON to the model.

Description & Review Guide

• What are the major changes?
  • Preserve the complete JSON-safe MCP CallToolResult, including successful and error payloads, under a private core-owned additional_properties marker.
  • Bound retained Host payloads to 1 MiB per result by default using a bounded preflight, with a per-MCP-tool configuration override; oversized Host data is omitted without changing the model result.
  • Bound cumulative Host JSON in complete AG-UI history to 8 MiB, retaining newest payloads while preserving private lossless model replay data.
  • Project that Host payload consistently through ordinary and approval-resolved live TOOL_CALL_RESULT events, MESSAGES_SNAPSHOT, and the supported agent_framework_messages_to_agui_host_history() conversion API.
  • Retain the exact model-facing result separately when AG-UI snapshots are replayed, with a content-only fallback if a client strips the private sidecar.
  • Preserve existing explicitly authored TOOL_RESULT_DISPLAY_KEY behavior and compatibility with supported core versions that predate the marker.
  • Add parser, error, live event, snapshot, replay, history, and payload-budget regression coverage and document the transport contract in the Python function-calling specification.
• What is the impact of these changes?
  • MCP widgets retain their structured hydration payload during streaming replacement and session reloads, while model context continues to use the independently selected model result.
  • Untrusted MCP servers cannot make AG-UI snapshots retain unbounded duplicate Host payloads.
  • Applications can use a supported Host-history converter while AGUIChatClient outbound requests remain model-safe.
  • Existing non-MCP tool results and standalone display markers keep their current behavior.
• What do you want reviewers to focus on?
  • The private core-to-AG-UI marker contract, bounded Host retention, the model-content replay sidecar, and the precedence of explicitly authored display payloads.

Related Issue

Fixes #7959

Draft PR #7897 addresses #7866's model-content selection and deduplication; it does not preserve Host/UI payloads and is intentionally not duplicated here.

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change. If it is breaking, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

[Eduard van Valkenburg (eavanvalkenburg)]

likebean HaoFeng Zhao (@ronronner02) Shivani . (@Shivani767) This draft follows the MCP result discussions in #7959 and #7897: it leaves #7866 model-content selection/deduplication to #7897 and adds the orthogonal core-to-AG-UI contract that preserves the full Host payload through live events, MESSAGES_SNAPSHOT, errors/approvals, and history reload without replaying UI-only JSON to the model. Review of that payload and replay contract would be appreciated.

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/ag-ui/agent_framework_ag_ui
_agent_run.py	1494	148	90%	163, 223, 242, 289, 307, 323–324, 446, 499–500, 507, 616, 620, 622, 639, 666–667, 723, 752, 766, 770, 774, 777, 782, 787, 799, 806–812, 823, 837, 844, 854, 869, 872, 904, 939, 963, 976, 1025, 1043, 1045, 1071, 1073, 1075, 1078–1082, 1106, 1137, 1173, 1228–1229, 1271, 1285, 1330–1336, 1345, 1353, 1381, 1403, 1407, 1410, 1412, 1472–1473, 1479–1485, 1588, 1592–1593, 1596, 1601–1602, 1605, 1611–1612, 1616–1618, 1636–1638, 1663–1664, 1687–1688, 1701, 1743–1745, 1756–1758, 1760, 1774–1776, 1904, 1912, 1942, 1946, 1997, 2165–2166, 2205, 2263, 2372, 2377, 2449, 2451, 2458, 2474, 2479, 2536, 2541, 2545, 2594, 2645, 2720, 2866, 2904, 2912, 2914, 2917, 2923, 2978, 2981, 2991–2992, 2999, 3046
_message_adapters.py	635	48	92%	175–176, 185–187, 190, 193–197, 199–204, 207, 216–218, 221–224, 260, 428, 549–551, 608, 611, 613, 616, 619, 635, 652, 674, 774, 790–791, 869, 891, 961, 996–997, 1073, 1116
_run_common.py	665	32	95%	108, 183–184, 186, 188, 191, 197, 199, 213, 220, 244–245, 258, 277, 300–302, 313, 357, 369, 377, 380, 568, 589, 744, 879–880, 1031, 1208–1209, 1275, 1278
_utils.py	124	10	91%	92, 100, 104–106, 118, 122–123, 126, 146
packages/core/agent_framework
_mcp.py	1417	117	91%	137, 153, 308, 314, 423, 442, 663, 751–752, 871, 896, 947, 1062, 1065, 1075, 1079, 1126–1127, 1132, 1139–1140, 1147, 1152–1153, 1160–1161, 1165, 1170–1171, 1180, 1187–1188, 1206, 1219, 1243–1244, 1263–1266, 1268–1269, 1273, 1299, 1333–1335, 1337, 1391–1393, 1452–1453, 1736, 1777–1778, 1791, 1794, 1803–1804, 1809–1810, 1816, 1870–1871, 1891–1892, 1901–1902, 1907–1908, 1914, 1992, 1995, 2022, 2045–2049, 2072–2074, 2079, 2083–2084, 2194, 2201, 2203, 2279, 2294–2295, 2302–2303, 2308–2309, 2314, 2321, 2336, 2398, 2584, 2586, 2608, 2610–2613, 2626–2627, 2671, 2733, 3195–3196, 3241, 3472–3473, 3491
_tools.py	1535	98	93%	232–233, 410, 412, 425, 450–452, 460, 478, 492, 499, 506, 529, 531, 538, 546, 681, 715–717, 720–722, 724, 730, 785–787, 813, 839, 843, 881–883, 887, 1060, 1072, 1079–1082, 1103, 1111, 1125–1127, 1506, 1591, 1704–1705, 1762, 1809, 1816–1817, 1921, 1998, 2094, 2108, 2111, 2118, 2121, 2127, 2139, 2156, 2165, 2173, 2177, 2197, 2199, 2206, 2264, 2267, 2290, 2297, 2302–2303, 2306, 2310, 2313, 2335, 2369, 2437, 2466–2467, 2564, 2592, 2632, 2635, 2692, 2845, 2954, 3127, 3130, 3140, 3157–3158, 3626
TOTAL	48492	4512	90%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
9862	36 💤	0 ❌	0 🔥	2m 43s ⏱️

[Copilot]

Pull request overview

Preserves complete MCP Host payloads across AG-UI events, snapshots, approvals, and history replay while retaining model-facing results separately.

Changes:

• Adds a core MCP Host-payload marker, including error propagation.
• Projects payloads through AG-UI live, snapshot, approval, and replay paths.
• Adds regression tests and documents the transport contract.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
python/packages/core/agent_framework/_mcp.py	Preserves MCP Host payloads.
python/packages/core/agent_framework/_tools.py	Propagates exception metadata.
python/packages/core/tests/core/test_mcp.py	Tests payload persistence and errors.
python/packages/ag-ui/agent_framework_ag_ui/_utils.py	Adds payload extraction and replay helpers.
python/packages/ag-ui/agent_framework_ag_ui/_run_common.py	Projects live and snapshot payloads.
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py	Converts persisted payloads and model sidecars.
python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py	Handles approval-resolved payloads.
python/packages/ag-ui/tests/ag_ui/test_run_common.py	Tests events, snapshots, and approvals.
python/packages/ag-ui/tests/ag_ui/test_message_adapters.py	Tests history reload and fallback.
docs/specs/004-python-function-calling-loop.md	Documents the MCP Host contract.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Eduard van Valkenburg (eavanvalkenburg)]

/review

[github-actions]

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): 118b5673f3f4
Model: gpt-5.6-sol-fast

Overview

The change cleanly separates complete MCP Host payloads from model-facing results, attaches the payload only once, and adds provenance, safe error fallback, rich-content replay, approval, and outbound-isolation guards. Two Medium risks remain: complete untrusted Host payloads are retained without a byte bound, and model-item serialization is eagerly performed for ordinary non-MCP results even though its output is discarded.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (2 medium) across 2 files. Details are attached to the affected lines below.

Affected areas: python/packages/ag-ui/agent_framework_ag_ui/_run_common.py, python/packages/core/agent_framework/_mcp.py

[github-actions]

MAF Automated Review — Iteration 2

Result: Findings reported
Scope: full PR (1 commit(s)): b84adcf3b867
Model: gpt-5.6-sol-fast

Overview

The change cleanly separates bounded MCP Host payloads from model-facing results and adds strong coverage for custom parsers, errors, rich content, approvals, and model-safe outbound requests. The marker is attached once, replay strips it from model items, and ordinary tools avoid the new serialization path. Residual risk remains because oversized payloads are fully copied before the cap is checked, retained payloads have no cumulative history budget, and the persisted-core-Message conversion path is not wired into a supported production surface.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
3 verified findings remained after source verification (3 medium) across 3 files. Details are attached to the affected lines below.

Affected areas: python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py, python/packages/ag-ui/agent_framework_ag_ui/_run_common.py, python/packages/core/agent_framework/_mcp.py

[Evan Mattson (moonbox3)]

/review

[github-actions]

MAF Automated Review — Iteration 3

Result: Findings reported
Scope: full PR (1 commit(s)): 76f2417562b1
Model: gpt-5.6-sol-fast

Overview

The change cleanly separates bounded MCP Host payloads from model-facing results, preserves rich replay content, and adds coverage for ordinary, approval, error, snapshot, and persisted-history paths. Its per-result preflight, model-safe outbound conversion, provenance marker, error redaction, and newest-first Host-content budget address the previously reported data-loss and direct payload-growth risks. Two residual issues remain: server-controlled metadata can expand into an unbudgeted replay sidecar large enough to exhaust shared process resources, and the new constructor option shifts established positional parameters.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (1 high, 1 medium) across 2 files. Details are attached to the affected lines below.

Affected areas: python/packages/ag-ui/agent_framework_ag_ui/_utils.py, python/packages/core/agent_framework/_mcp.py

[github-actions]

The 8 MiB history cap does not cover this replay sidecar, and to_dict() retains each item's server-controlled _meta. Because core stamps the same _meta on every parsed MCP item, a sub-1 MiB result with many items can expand to hundreds of MiB in _agentFrameworkModelContent before _bound_host_payload_history() measures only content, exhausting shared process memory/bandwidth and persisting outside the cap. Please bound the serialized sidecar as part of the aggregate budget or avoid replicating _meta while preserving the model-facing replay data.

[github-actions]

max_host_payload_size_bytes was inserted before existing positional parameters. Third-party MCPTool subclasses or concrete-tool callers that passed additional_properties (or any later option) positionally now bind that value to this parameter and can fail during construction at max_host_payload_size_bytes <= 0 (for example, a dictionary raises TypeError). Please preserve the established positional ordering by appending this option after existing parameters, or introduce it as keyword-only without shifting them.