Responses backend rejects plaintext reasoning after cross-provider routing

[srchandrupatla]

Problem

When a stage router switches between a local OpenAI Responses-compatible model and a strict OpenAI Responses backend, provider-specific reasoning items can make the next target reject otherwise valid conversation history.

Local to strict OpenAI

Some local servers, including llama.cpp, return reasoning items with plaintext content and an empty encrypted_content. Replaying that history to the strict backend fails with:

Invalid 'input[3].content': array too long. Expected an array with maximum length 0, but got an array with length 1 instead.

Strict OpenAI to local

The reverse handoff has a separate incompatibility. A signed, encrypted reasoning item normalized for strict OpenAI cannot be consumed by llama.cpp:

item['content'] is not an array

Adding an empty array is also rejected:

item['content'] is empty

Expected behavior

Switchyard should normalize Responses reasoning items for the selected target while preserving messages, function calls, and function-call outputs:

• With the default responses_reasoning = "preserve_encrypted" policy, remove unsigned/plaintext reasoning items and retain signed encrypted reasoning with an empty plaintext content array.
• With an explicit responses_reasoning = "drop" policy, remove all reasoning items for a local backend that cannot consume another provider's encrypted representation.
• Preserve the remaining conversation and tool-call history in order.

The policy is explicit rather than inferred from model names, URLs, or authentication, so authenticated local servers and unauthenticated hosted-compatible servers can be configured correctly.

Implementation and validation

A focused implementation is available on the fork branch:

https://github.com/srchandrupatla/Switchyard/tree/fix/responses-reasoning-handoff

Relevant commits:

• 30c648c2 — sanitize unsigned Responses reasoning handoffs
• ab9b9e8f — add the explicit, model-agnostic Responses reasoning replay policy
• 0971862f — keep the existing public backend configuration API source-compatible

Validation completed so far:

• Formatting and Clippy pass.
• 55 switchyard-llm-client unit tests pass.
• 11 client observability tests pass.
• The full non-PyO3 Rust workspace test suite passes.
• Dedicated HTTP-level regression tests cover both handoff directions.
• A release build on macOS passed a real llama.cpp cloud-to-local request with HTTP 200.
• The reverse local-to-gpt-5.6-sol request also completed successfully.
• Two isolated 20-request bidirectional stress runs completed with 20/20 successes.

I am continuing production stress testing before opening a PR.