Revisit structured output semantics: parse source and the run() failure boundary

[shibayan]

Summary

Two coupled structured-output decisions are currently deliberate but observable. They must be decided independently and then recorded together.

1. Parse source: whole response versus final assistant message

applyStructuredOutput reads response.text, the concatenation of text from every response message. That mirrors .NET's AgentResponse<T> behavior, but Python changed to the last non-empty assistant message as an explicit bug fix and Go independently parses only the final message buffer.

Whole-response parsing fails when a tool round contains assistant prose before the final JSON, and it can return an earlier JSON object instead of the final answer.

Decision: retain .NET-first behavior and document the counterexamples, or follow the Python/Go final-message behavior with one shared helper used by agent and direct chat-client paths.

2. Failure boundary: persisted success followed by run() rejection

The turn is persisted successfully before parsing/validation. A parse failure then rejects agent.run(), hiding the raw response, usage, and metadata from the caller. .NET and Python parse lazily on typed-value access; Go fails the run before its default history provider persists it. The current split is TypeScript-specific and deliberate.

Decision: retain eager rejection and document it, or return the completed raw response and surface the parse/validation error only when its typed value is requested. An async getValue() is required if Standard Schema validation must become genuinely lazy.

Acceptance criteria

• Parse source and failure boundary are decided independently; this issue is not closed after deciding only one half.
• Both outcomes are recorded with rationale, tests, and user-visible migration guidance for any behavior change.

Parse source

• Tests cover assistant prose before a tool call followed by final JSON, an earlier valid JSON value followed by a different final value, and one top-level value plus trailing content within the selected message.
• Agent.run() and a direct client wrapped with structured output use the same contract.
• Awaited and fully consumed streaming paths fold to the same result.
• If final-message parsing is adopted, one shared helper selects the last non-empty assistant message and earlier tool-round text cannot win.
• If whole-response parsing is retained, the counterexamples are pinned and documented as intentional .NET-first behavior.

Failure boundary

• Invalid JSON, truncation with finishReason: "length", synchronous validation failure, and asynchronous Standard Schema refinement failure are covered.
• The transcript is persisted exactly once on validation failure; suspended responses keep the existing no-parse behavior.
• If lazy failure is adopted, the API defines value, async getValue(), and error caching. Callers can inspect raw text, usage, finish reason, IDs, metadata, and raw representation before surfacing the validation error.
• If eager rejection is retained, tests pin the successful-session/rejected-run boundary and public documentation calls it out.
• Any public API change includes TSDoc, README, CHANGELOG, and public-api-change review.