Answer on Responses API models, and set how hard they think (#212)

[ethanstoner]

Closes #212.

What this changes

Two things, and the first is why the second was worth having.

The framework Bot answers on 5.6-tier models. #212 asks to run gpt-5.6-luna on the Responses API Bot. Today that Bot starts, reports healthy, and says nothing — RUN_STARTED, RUN_FINISHED, no text. This repository already documents it, in .env.example and on the agent-langgraph service in docker-compose.yml:

This integration answers nothing at all on gpt-5.6-* through the Responses API: RUN_STARTED, then RUN_FINISHED, no text. Driven against the real service.

It is one line in the stream loop:

const text = typeof chunk?.content === "string" ? chunk.content : "";
if (!text) continue;

Chat completions streams content as a string, so that holds on 5.5. The Responses API does not. @langchain/openai turns every response.output_text.delta into a content block — its own convertResponsesDeltaToChatGenerationChunk builds [{ type: "text", text: delta, index }]. So on 5.6 the condition was false for every delta, text was "" every time, continue ran every time, and the run finished having emitted no TEXT_MESSAGE_CONTENT at all. Both shapes are read now.

Only type: "text" blocks are read. A reasoning model streams its summary into the same array under a different type; that is the Bot's private working rather than its answer, and a surface printing it would show the person something never meant for them. There is a test for exactly that.

BOT_REASONING_EFFORT, which is what the issue asked for: none, minimal, low, medium, high, xhigh, max — the list the installed API types carry. Sent as reasoning: { effort }, not the reasoningEffort convenience field the integration deprecated in favour of merging into that object. Unset sends nothing and the model keeps its provider's default.

Validated at startup rather than passed onward, which is the other half of the request. An effort the API does not have is dropped somewhere down the stack, and a Bot that starts, looks configured and thinks for as long as it likes is worse than one that refuses and says why. Same for the two ways this setting reaches an API with nowhere to put it — a provider that is not OpenAI, and a model not on the Responses API. All three messages name the variable that would fix it.

Where it runs

• New state that outlives a request? None. Three constants read from the environment at startup and one pure function called inside the existing stream loop.
• What happens on the second replica? It reads the same environment and behaves identically. There is nothing to disagree about: no cache, no counter, no first-writer-wins.
• Anything serialised? No.
• Anything fanned out to a browser? Only through the AG-UI stream the run already owns, on the socket that made the request. This adds TEXT_MESSAGE_CONTENT events that should have been there, and no new channel.
• New listener, port, or schedule? None.

Boundary and audit

• Every acting call still goes through the gateway: untouched. The tool loop, callTool, and the run assertion are unchanged.
• New refusals and new failures each write a row: no new runtime refusal. The three new refusals are startup configuration errors that stop the process before it serves anything, matching how BOT_PROVIDER and a missing model key already behave.
• Nothing new is trusted from the client: the new setting comes from the deployment's environment. Nothing was added to what a request may say.

Changelog

• A line in CHANGELOG.md under Unreleased.

Proof

The real entrypoint, driven against a stand-in for OpenAI's Responses API emitting the documented SSE events, so the whole path runs — including the integration's own parsing. No key needed, and the upstream request is captured so it can be asserted on.

Before, BOT_MODEL=gpt-5.6-luna:

http 200
events: RUN_STARTED -> RUN_FINISHED
text deltas: []
RESULT: the Bot answered NOTHING (RUN_STARTED then RUN_FINISHED, no text)

After, same command:

http 200
events: RUN_STARTED -> TEXT_MESSAGE_START -> TEXT_MESSAGE_CONTENT -> TEXT_MESSAGE_CONTENT ->
        TEXT_MESSAGE_CONTENT -> TEXT_MESSAGE_CONTENT -> TEXT_MESSAGE_CONTENT ->
        TEXT_MESSAGE_CONTENT -> TEXT_MESSAGE_END -> RUN_FINISHED
text deltas: ["Hello"," from"," the"," Responses"," API","."]
RESULT: the Bot answered

With BOT_REASONING_EFFORT=high, the upstream request carried it:

{"model":"gpt-5.6-luna","stream":true,"reasoning":{"effort":"high"}, ...}

No regression on the older path. BOT_MODEL=gpt-5.5 against a chat-completions stand-in still reports responsesApi: false and still streams all six deltas.

The three refusals, each run, each exiting 1:

$ BOT_MODEL=gpt-5.6-luna BOT_REASONING_EFFORT=maximum bun src/index.ts
BOT_REASONING_EFFORT=maximum is not an effort this API has. Use one of: none, minimal, low, medium, high, xhigh, max.

$ BOT_PROVIDER=anthropic BOT_REASONING_EFFORT=high bun src/index.ts
BOT_REASONING_EFFORT is OpenAI's setting, and BOT_PROVIDER=anthropic. Unset it, or set BOT_PROVIDER=openai.

$ BOT_MODEL=gpt-5.5 BOT_REASONING_EFFORT=high bun src/index.ts
BOT_REASONING_EFFORT needs the Responses API, and BOT_MODEL=gpt-5.5 is not being run on it. Use a model that requires it, or set BOT_RESPONSES_API=true.

Suite: agent-langgraph and agent-bot pass (15 tests, 4 new). I could not get a number out of the full test:ci run locally, so I am not claiming one: with a pgvector:pg17 container and migrations applied it sat at near-zero CPU for tens of minutes without producing output, across three attempts. That looks like a Windows-host problem rather than anything in this diff — the change is three constants and one pure function in agent-langgraph, which is not in the workspace the DB suite covers. CI will be the real check. bun run typecheck clean. agent-langgraph has no typecheck script and is not in the deployables matrix, so I ran tsc --noEmit over src/ and tests/ by hand — clean. Happy to add it to that matrix in a separate PR if you want the coverage; it looked out of scope here.

biome format and biome lint are clean on the five files this touches. (Checked against LF copies — a Windows checkout with core.autocrlf=true fails format:check on 442 of 444 files, mine included, which is a line-ending artifact rather than anything in the diff.)

Notes

• The docs that recorded the old behaviour are updated rather than left contradicting the code: the .env.example note, the docker-compose.yml service comment, and a BOT_REASONING_EFFORT entry in both.
• agent-bot, the hand-written chat-completions Bot, is deliberately untouched. The issue asks that it stay on a compatible model, and it already refuses to start on one whose tools it cannot use.