[api][python] Add explicit output schema parameter to the chat path (structured-output foundation)

[weiqingy]

Linked issue: #280

Purpose of change

This is the foundation for native structured output, split out from the combined foundation-plus-OpenAI PR so it can be reviewed on its own; the OpenAI native translation follows in #919 (stacked on this PR).

It carries the output schema to chat model connections as an explicit parameter rather than an implicit entry in the provider-facing model parameters, and splits the structured-output strategy into a user policy and a per-model capability.

Java adds a concrete four-argument chat overload whose default body ignores the schema and delegates to the existing abstract three-argument chat, so connections with no native mechanism need no change. Python adds an explicit output_schema parameter to every connection, including the Java-bridge connection, so the schema binds to a named parameter and cannot be forwarded into a provider request through **kwargs.

StructuredOutputStrategy (AUTO/NATIVE/PROMPT, default AUTO) expresses user intent on the setup. Capability is separate: a connection-side predicate over the effective model, since a provider supports its native mechanism only on some models. The base predicate returns false; a connection that adds a native translation overrides it.

This is the mechanism only. No connection applies native structured output and no path passes a schema on the chat call yet, so behavior is unchanged. The policy resolver and the capability predicate have no caller in this PR by design; the first consumer is #919 (the stacked OpenAI PR), and wiring structured output onto an agent's final output is a separate follow-up (#912).

Tests

Java: BaseChatModelConnection overload delegation and the default capability predicate.

Python: the explicit parameter on every connection (a subclass-walk guard proves none can leak a schema into **kwargs), the policy default, and strategy resolution.

./tools/ut.sh -j and the Python suite pass; lint and license checks are clean.

API

Yes. Adds the chat overload / output_schema parameter, the StructuredOutputStrategy enum on the setup, and the capability predicate on the connection, in both Java and Python.

Documentation

• doc-needed
• doc-not-needed
• doc-included

[weiqingy]

The failing check was unrelated to this PR — it was a transient CI infra failure:

• it-java [ubuntu-latest] [java-17] [flink-2.3]: The Install java setup step failed before any test ran — actions/setup-java@v4 got an HTTP 403 downloading the Temurin JDK 17 tarball from Adoptium's release server, so the actual IT steps were skipped. The other it-java matrix legs (flink-2.0/2.1/2.2) passed in the same run. Evidence: failed job.

Re-triggered CI (empty commit) to clear it — all checks are green now.

[GreatEugenius]

Hi @weiqingy, thanks for the PR. Overall, the approach looks good.

One thought on the strip: right now every non-native connection has to pop the key itself (the 4 non-OpenAI Python ones do it purely to avoid leaking; the Java ones don't strip at all and are only safe incidentally). Since both languages already funnel through BaseChatModelSetup.chat() before connection.chat(...), could we strip there based on the flag?

# Python
if not connection.supports_native_structured_output:
    merged_kwargs.pop(STRUCTURED_OUTPUT_SCHEMA_KEY, None)

// Java
if (!connection.supportsNativeStructuredOutput()) {
    params.remove(BaseChatModelConnection.STRUCTURED_OUTPUT_SCHEMA_KEY);
}

That removes the 4 leak-only pops on the Python side, closes the Java asymmetry, and gives the flag a real caller on both. Native connections would still pop themselves (to discard when tools are bound). Not blocking.

[weiqingy]

Thanks for the review, @GreatEugenius — this is a nice simplification. Done in f35e432.

[GreatEugenius]

LGTM. The centralized strip in BaseChatModelSetup.chat reads cleanly, and I believe this lays a solid foundation for the follow-up wiring (ReActAgent final-output). Thanks @weiqingy for the quick turnaround — looking forward to the rest of the stack! @wenjin272, could you please take a look?

[weiqingy]

Thanks, @GreatEugenius!

@wenjin272, could you take a look when you get a chance?

[wenjin272]

Thanks for taking this on, @weiqingy.

Passing outputSchema through modelParams and using hasTools as an indirect way to exclude ReAct loop calls both feel somewhat implicit to me, because modelParams otherwise represents provider-facing generation parameters, while outputSchema is framework-level execution metadata. This requires a reserved key and coordinated stripping or consumption across the setup and connection layers, and the structured-output behavior is not discoverable from the chat() method signature.

Would it make sense to introduce an explicit structured-output API instead or provide a model.with_structured_output like langChain? It may also be helpful to look at how other agent frameworks model this.

I also think we should first clarify how structured output should apply only to an agent’s final output, since that decision could affect the design of the underlying LLM/chat model interface.

[wenjin272]

According to the OpenAI API documentation, only gpt-4o-mini, gpt-4o-2024-08-06, and later models support structured outputs. However, since the DEFAULT_MODEL in OpenAICompletionsSetup is set to gpt-3.5-turbo, requests using the default configuration will fail when sending response_format: json_schema, rather than gracefully falling back to the prompt.

I think we can move supportsNativeStructuredOutput to ChatModelSetup, or alternatively, change the DEFAULT_MODEL.

[weiqingy]

Good catch, conceded.

One thing to add: the user can't recover from this failure. ChatModelAction retries (re-failing deterministically, since a 400 for an unsupported model isn't transient), then applies ErrorHandlingStrategy, so the user gets a thrown error or, under IGNORE, a silently dropped record. Nothing falls back to prompt. It slipped past the tests because OpenAICompletionsConnectionTest pins "model", "gpt-4o", so the native test only passes on a capable model.

On the fix, I'd split the flag rather than move it. A ChatModelSetup boolean isn't enough alone: BaseChatModelSetup.chat() merges per-call modelParams over the setup's, so model is overridable per request (nothing does that today, but the interface allows it), and the connection has its own defaultModel under that. So policy (auto/native/prompt) on the setup, and capability as a connection-side check against the effective model at request-build time. Under auto, an incapable model falls back to prompt instead of failing.

The stale gpt-3.5-turbo default looks like a separate bug anyway (the javadoc example already uses gpt-4o-mini, and OpenAIResponsesModelSetup defaults to gpt-4o), so I'll spin that out on its own.

[weiqingy]

Thanks @wenjin272, I agree with both objections and the order you're suggesting.

modelParams as the schema channel. It's the wrong container: modelParams is provider-facing generation params, the schema is framework execution metadata. Routing it through there costs a reserved key, a strip in BaseChatModelSetup.chat(), a pop in the native connection (each in both languages), and three test files that only prove the key never reaches an SDK. All it buys is an unchanged chat() signature, which is thin at 0.3-SNAPSHOT.

I'll also retract a claim from my own design doc: that an explicit parameter would break every connection. It won't. In Java a defaulted overload chat(messages, tools, modelParams, outputSchema) needs no edits to non-native connections. In Python it's one named output_schema=None per connection, and that parameter is exactly what kills the leak, since today the schema rides inside the **kwargs forwarded to the SDK.

hasTools. The bigger issue is that it makes the feature inert: BaseChatModelSetup binds tools once in open() and passes them on every call, so the native path never fires for a ReActAgent that has both tools and an output_schema, which is the case it's for. hasTools is a proxy for "is this the final turn," and only the caller knows that.

The root cause is the missing finalization step. The gate only exists because there's no post-loop call. Make final-output structuring a separate call (full history, schema bound, no tools) and both problems disappear: the caller declares when structure is wanted, and the schema is a real argument. That ordering removes machinery instead of adding it.

Proposed shape:

1. Explicit schema on the chat path (deletes the reserved key, strip, pop, and no-leak tests).

2. Finalization as a dedicated post-loop call at ChatModelAction's no-tool-calls branch. One caveat: the LangChain doc you linked has moved past the post-loop-call shape and now uses ProviderStrategy (native) plus ToolStrategy (a synthetic tool called inside the loop, no extra call). So the extra round-trip is a cost that prior art avoids, which is my first question below.

3. Split policy from capability: the setup declares policy (auto/native/prompt), the connection checks capability against the effective model at request-build time. Under auto, an incapable model falls back to prompt instead of failing.

On with_structured_output itself: it returns a per-call object, but our ChatModelSetup is a named Resource, open()-ed once and re-resolved by name per record. A derived Setup has no registry name and can't cross the Pemja bridge; mutating the shared one is a data race in a streaming operator. The part that ports is the declarative strategy selection, which is where (3) comes from. Open to a different shape if you had one in mind.

A few calls where I have a lean but want your read:

1. The post-loop call costs one extra LLM round-trip per record. I'd start with that, since it's simpler and the parse side already assumes a no-tool-calls final turn, and treat the forced-tool alternative (ToolStrategy) as a later optimization. The mechanism exists on-repo (AnthropicChatModelConnection strict tool use, :186-200) but doesn't force the call, so it'd be new behavior.

2. For how auto decides capability, I'd avoid a known-not-supported list (it rots on every provider release) and lean toward native only when the user explicitly configures a capable model, prompt otherwise.

3. I'd keep finalization as its own follow-up PR (a [Feature] Support making use of the native capabilities of models for structure output. #280 sub-issue), as we'd agreed, leaving this PR the foundation. Let me know if folding it in would make the review easier.

WDYT?

[wenjin272]

Thanks @weiqingy , this direction makes sense to me, and I agree with the proposed approach and sequencing.

One point to keep in mind for the finalization follow-up: the current ReAct implementation automatically adds structured-output instructions to the prompt whenever the user declares an output_schema. If the finalization call uses a model with native structured-output support, that prompt augmentation is unnecessary and could result in two overlapping schema channels.

The finalization path should therefore select the strategy before constructing the prompt: add schema instructions only for the prompt-based fallback, and omit them when native structured output is used.

[weiqingy]

Thanks @wenjin272.

Good catch. One nuance: today that schema instruction is registered unconditionally at agent construction, in both languages (react_agent.py:136-139, ReActAgent.java:81-86), so folding your point in means making that registration strategy-driven — emit it only on the prompt-based fallback, and omit it when the native path carries the schema through response_format. That reinforces the policy/capability split: one strategy decision gates both, so the two channels can't disagree.

I'll capture it as a requirement on the finalization sub-issue.

[weiqingy]

@wenjin272 The rework implementing the explicit-output-schema approach is up. I split it into two stacked PRs so each is one logical change and easier to review:

• [api][python] Add explicit output schema parameter to the chat path (structured-output foundation) #843 (this one) is now the foundation only: the explicit output_schema parameter plus the policy/capability split. It's mechanism-only — nothing threads a schema on the chat path yet, so behavior is unchanged.
• [integrations][openai] Apply OpenAI native structured output #919 stacks on top with the OpenAI native response_format translation.

The remaining providers (Azure, Ollama, Anthropic, DashScope) will follow as their own PRs, and applying structured output to an agent's final output remains the separate follow-up (#912).

The policy resolver and capability predicate have no caller in #843 by design — #919 is their first consumer.

Could you take a look when you have a chance? Thanks!

[wenjin272]

Thanks for the updates. I left three comments.

[wenjin272]

It appears that the BaseChatModelSetup has never invoked this chat interface. Since ChatModelAction actually calls BaseChatModelSetup#chat, should we also add the corresponding interface to BaseChatModelSetup?

[weiqingy]

Good question, and yes, nothing calls the four-argument chat in this PR.

That is the scope here: the mechanism, with no framework path passing a schema yet. The setup-level overload is the point where the policy on the setup meets the connection-side capability, and where that resolution should live is still open. #912 lists it as an open question: "Where should auto resolve policy into a concrete strategy, given capability is a connection-side predicate over the effective model at request-build time?"

#912 is also the consumer. It issues a dedicated structured call at ChatModelAction's no-tool-calls branch, so it is the change that both needs the setup-level entry point and can actually exercise it. Adding the overload here would settle that open question with nothing able to validate the choice, and it has to move together with normalizing effective-model resolution across Java and Python.

If you would rather see the complete path in one PR, I am happy to pull it forward here instead.

[wenjin272]

make sense

[wenjin272]

When NATIVE is configured, resolvesToNative(false) still returns true, so the caller may invoke the four-argument chat() even when the connection does not support native structured output. With the current default implementation, outputSchema is silently ignored and an unconstrained response may be returned. I think the default implementation should throw UnsupportedOperationException for a non-null outputSchema; supporting connections can override it, while null can continue delegating to the three-argument overload.

[weiqingy]

Agreed, fixed in aead976.

The default overload now throws UnsupportedOperationException for a non-null outputSchema and still delegates to the three-argument overload for null, so a connection that translates a schema overrides it and nothing changes for schemaless calls.

Python needed a different shape for the same contract. chat is abstract there, so there is no inherited body that could refuse a schema. A guard on the connection base raises NotImplementedError, and every connection without a native translation calls it as the first statement of chat, before any client access. The connection subclass walk in test_output_schema_param_declared.py now asserts the rejection as well, so a connection added later is held to the same contract.

One thing worth flagging since it is not visible from the diff: the throw has no production caller in either language yet, so it stays dormant until the native path lands. Java's only connection call is the three-argument one in BaseChatModelSetup, and on the Python side chat_model_action.py calls chat without a schema and applies the prompt fallback separately.

[wenjin272]

There seems to be a Java/Python parity issue for an explicitly configured structured_output_strategy: null: Java treats it as AUTO, while Python rejects None with a validation error. I think we should either normalize null to AUTO in both languages or reject it consistently.

[weiqingy]

Good catch, fixed in e537e4d.

Python now normalizes an explicit null to AUTO, matching Java. I went that direction rather than rejecting in both because Java cannot reject it: getArgument is a plain map lookup (ResourceDescriptor.java:93-96), so an absent key and an explicit null reach fromArgument identically, and rejecting the one would reject the other.

Only null normalizes. An empty string, 0, False and an unrecognized name all still raise, and the case-insensitive resolution of both the lowercase values and the Java enum names is unchanged.

I also added the Java test for an explicitly null argument. The neighbouring strategy tests build their descriptors with Map.of, which rejects a null value, so that case could not be expressed there and the behavior this parity fix relies on was not pinned on the Java side.

[wenjin272]

Thanks for addressing my comments. LGTM