Run every Bot on a current model - #168
Merged
Merged
Conversation
The example package shipped `gpt-4.1` as the default model for every built-in Bot. That is not a tuning choice, it is the cause of a bug I spent the afternoon attributing to the prompt. Asked to open a page behind a sign-in, the built-in Bots answered "would you like me to prompt you to sign in?" and called nothing, three times out of three. `COMPUTER_GUIDANCE` forbids that sentence in as many words, so I went looking for a prompt lever, and there was none to find: on gpt-5.6-terra the same question produces `computer_request_help`, with `requested: true` and the reason on the row, first try. The remote Bot had been getting this right all along because it runs a newer model. So: gpt-5.6-terra everywhere it can go. Of the three 5.6 tiers, terra is the like-for-like replacement for 5.5 rather than an upgrade in spend, and sol is there for a deployment that wants it. The Responses API is now inferred from the model rather than left to a separate switch. gpt-5.6-* rejects function tools on chat completions, so a deployment that set BOT_MODEL and did not also know about BOT_RESPONSES_API got a Bot that started, looked healthy, and failed on its first tool call. The switch is still honoured for a model this build has not heard of. agent-bot stays on gpt-5.5 and says why. It speaks /v1/chat/completions by hand, and the endpoint offers exactly two ways to use 5.6 there: the Responses API, which is the streaming-loop rewrite this file exists to avoid, or reasoning_effort 'none'. Turning reasoning off on the Bot whose whole job today is deciding when to ask a person for help would be trading the fix for the bug. Also swept: two `gpt-4o` references in .env.example and the configuration docs that predate all of this, and the gpt-4.1 fixtures in the tests.
davidmckayv
requested review from
MikeRyanDev,
guidovizoso and
tylerslaton
as code owners
August 22, 2026 15:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The example package shipped
gpt-4.1as the default model for every built-in Bot.That is not a tuning choice. It is the cause of a bug I spent the afternoon attributing to the prompt.
What it was doing
Asked, in a fresh channel, a neutral question — "Open drive.google.com and tell me the name of the first file listed in my Drive" — the built-in Bots answered:
and called nothing. Three times out of three. No
computer_request_help, sorequestedstayedfalse, no banner, no button.COMPUTER_GUIDANCEforbids that exact sentence in as many words, so I went looking for a prompt lever. There wasn't one. Ongpt-5.6-terrathe same question produces the tool call first try:{"holder":"bot","requested":true, "reason":"Google Drive is showing a sign-in page; sign in to your account so I can read the first listed file."}The remote Bot had been getting this right all along — because it runs a newer model.
Which model
The 5.6 family is three tiers, and there is no plain
gpt-5.6: sol (flagship, most reasoning), terra (balanced, 5.5-competitive at half the cost), luna (fastest, low-reasoning).terra is the default here: it is the like-for-like replacement for today's 5.5 rather than an upgrade in spend, and
BOT_MODELpicks sol for a deployment that wants it. Luna would be the wrong default for Bots whose job includes deciding when to call a tool.The Responses API is now inferred
gpt-5.6-*rejects function tools on/v1/chat/completions:So a deployment that set
BOT_MODELto a 5.6 model and did not also know aboutBOT_RESPONSES_APIgot a Bot that started, looked healthy, and failed on its first tool call. It is now inferred from the model inagent-langgraphand both example Bots; the switch is still honoured for a model this build has not heard of.agent-bot stays on 5.5, and says why
It speaks
/v1/chat/completionsby hand. The endpoint offers exactly two ways to use 5.6 there: the Responses API, which is the streaming-loop rewrite this file exists to avoid, orreasoning_effort: 'none'.Turning reasoning off on the Bot whose whole job today is deciding when to ask a person for help would be trading the fix for the bug. Verified both: all three 5.6 tiers do call tools on chat-completions with
reasoning_effort: 'none', and 5.5 calls them with reasoning intact.Swept
examples/fintech/model.yamlgpt-4.1gpt-5.6-terradocker-compose.yml×2gpt-5.5gpt-5.6-terraagent-langgraphdefaultgpt-5.5gpt-5.6-terra+ inferred Responses APIexamples/langgraph-botgpt-5.5gpt-5.6-terra+ inferred Responses APIexamples/mastra-botgpt-5.5gpt-5.6-terra+openai.responses().env.examplegpt-4o,gpt-5.5docs/configuration.mdgpt-4o×2,gpt-4.1gpt-4.1agent-botgpt-5.5Model ids checked against the live
/v1/modelslist rather than assumed.Lint, format, typecheck clean; 52 model-related tests pass.