Repository navigation
fix(llm): no temperature for new Claude models, hand DeepSeek its reasoning back (ATO-1397) - #1123
Draft
sosidudku1 wants to merge 5 commits into
Draft
sosidudku1 wants to merge 5 commits into
sosidudku1 wants to merge 5 commits into
Conversation
Claude Sonnet 5.5, Haiku 5.5 and the other new Claude models answer a
request that carries `temperature` with a 400, and every turn sent 0.2.
- model-params: Claude Opus 4.7+, Sonnet 5+, Haiku 5+ and every Fable
and Mythos model get the `forbidden` temperature rule, read from the
family and version in the id (direct, router-prefixed and Bedrock
spellings). Older Claude models keep the free rule.
- allowed-temperature: read a 400 that refuses the field itself
("`temperature` is deprecated for this model", "unsupported
parameter") next to the existing "only N is allowed" lesson.
- openai-provider: the one resend on a temperature lesson now also
covers a refused field: the model is remembered as taking no
temperature and the resend goes out without one, so an id the static
rule does not know recovers after one 400.
- A learned refusal also removes a temperature set through extraBody or
model params, in turns and image descriptions.
Linear: ATO-1397
DeepSeek's thinking mode answers the next request of a tool loop with a 400 unless the earlier assistant tool-call messages carry their `reasoning_content`. Cloud bodies never carried it: the cloud context journal drops reasoning, and the ATO-887 hand-back was limited to local template servers. - build-cloud-prompt: the current, un-replied macro-turn's call reasoning is put back on the outgoing turns from the transcript. The persisted journal and the text form are unchanged, and a provider that does not hand reasoning back sends the same body as before. - openai-native-messages: a `reasoningRequired` option hands reasoning back without the local token cap and, once any call carries reasoning, gives every assistant tool-call message the field (empty where the transcript kept none). - openai-provider: the option is on for DeepSeek's own host only (`api.deepseek.com`); other services may reject the unknown field and routers carry reasoning their own way. Linear: ATO-1397
…hanges nothing - model-params: the Claude id rule is anchored at the start of the id (after the router prefix, with an optional `us.` / `anthropic.` Bedrock prefix), so a fine-tune that only names Claude, such as `Qwen3-8B-Claude-Sonnet-5-Distill`, keeps its temperature. - openai-provider: a refused-temperature 400 for a model already recorded as taking none, on a body that carried no temperature, gets no resend, since the same body would go out again. Linear: ATO-1397
…in the window - openai-native-messages: under `reasoningRequired` every assistant tool-call message carries `reasoning_content`, an empty string where the call has none. Before, the empty field was added only when some call carried reasoning, so the first step of a follow-up turn sent the answered calls without it. A new `reasoningBudget` option caps what is handed back, newest calls first. - openai-build-body: the reasoning handed back is held to the room the cloud window has left after the same messages without reasoning, the tools, the reply reserve and the safety margin. The cloud prompt does not count reasoning, so a long thinking turn failed the capacity check instead of sending less of it. - Tests: first step of a follow-up turn, the budget, a tight window, and one path from session through buildPrompt to the body sent to api.deepseek.com. Linear: ATO-1397
…ck measures The reasoning budget was worked out from the request's own cap and raw tools, but the capacity check measures the finished body: `max_tokens` from extraBody or model params (it is not a reserved key) and the strict tools that actually go on the wire. A DeepSeek model with `max_tokens: 65536` on a 128k window could still fail the check. The body is now assembled once without any reasoning, measured with the same input, reply reserve and margin as `checkedBody`, and assembled again with what the window has left (less ten per cent for escaping) as the reasoning budget. Tests pin the size in a tight window and cover a model-params `max_tokens` and strict tools, each with no capacity throw and reasoning still sent. Linear: ATO-1397
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linear: ATO-1397.
Problem
Two providers fail with HTTP 400 out of the box (Desktop 0.0.8 / agent 0.7.0):
temperature, which these models no longer accept.reasoning_content.Cause
model-params.tsknew two temperature exceptions (OpenAI reasoning models, Kimi K3.1). Every Claude model got the default 0.2. The run-time retry from fix(llm): send each model the temperature it accepts (Kimi K3.1 = 1), retry once on a named temperature #1008 only read "only N is allowed", not a 400 that refuses the field itself.Change
Temperature (first commit):
model-params.ts: Claude Opus 4.7 and later, Sonnet 5 and later, Haiku 5 and later, and every Fable and Mythos model get theforbiddenrule. It is read from family and version at the start of the id, with or without a router or Bedrock prefix (anthropic/claude-sonnet-5,anthropic.claude-haiku-5-5,us.anthropic.claude-opus-5-5, dotted or dashed versions, date suffixes). Anchored at the start, so a fine-tune that only names Claude (Qwen3-8B-Claude-Sonnet-5-Distill) keeps its temperature. Older Claude models (Opus 4.6, Sonnet 4.6, Haiku 4.5, theclaude-3-5-sonnetnaming) keep the free rule.max_tokensis unchanged.allowed-temperature.ts: newparseTemperatureRefused/temperatureLessonFromError. A 400 that says temperature is deprecated, unsupported, unknown or not permitted (within 80 characters of the word) reads as "this model takes no temperature". A body that names the one allowed value keeps its existing meaning.openai-provider.ts: the existing one resend per completion (fix(llm): send each model the temperature it accepts (Kimi K3.1 = 1), retry once on a named temperature #1008) now covers both lessons. On a refused field the model is remembered as taking no temperature (rememberTemperatureForbidden) and the resend goes out without one. So a Claude id the static rule does not know yet recovers after one 400, unary and streamed. When the refusal is already on record and the body that failed carried no temperature, there is no resend: the same body would go out again.openai-build-body.ts,openai-describe-image.ts: a learned refusal also removes a temperature set throughextraBodyor model params, as a learned value already overrides them.resolveTemperature, so they stop sending 0 to these models too.DeepSeek reasoning (second commit):
build-cloud-prompt.ts: the current, un-replied macro-turn's call reasoning goes back on the outgoing turns, read from the transcript. The persisted journal and the text prompt do not change. A provider that does not hand reasoning back sends the same body as before.openai-native-messages.ts: newreasoningRequiredoption. It hands reasoning back without the 2048-token local cap. Every assistant tool-call message carries the field, empty where the transcript kept none (answered turns, later calls of a batch, reasoning left out by the budget). This includes the first step of a follow-up turn, when only answered calls are in the history. AreasoningBudgetoption caps it, newest calls first.openai-build-body.ts: the reasoning handed back is held to the room the cloud window has left. The body is assembled once without any reasoning, after the extraBody / model params merge (so amax_tokensset there counts) and with the tools as they go on the wire (strict tools included). It is measured exactly as the capacity check measures it, then assembled again with the remaining room, less 10% for JSON escaping, as the reasoning budget. The cloud prompt does not count reasoning, so without this a long thinking turn would fail the capacity check instead of sending less of it.openai-provider.ts/openai-build-body.ts: the option is on only for DeepSeek's own host (isDeepSeekHost,*.deepseek.com). Other OpenAI-compatible services may reject the unknown field, and routers such as OpenRouter carry reasoning their own way.Tests
model-params.test.ts: Claude models without sampling parameters (forbidden list, older models and Claude-named fine-tunes still free, learned refusal per model id).allowed-temperature.test.ts:parseTemperatureRefused,temperatureLessonFromError.openai-build-body-temperature.test.ts: no temperature for Claude 5.5 / Fable, 0.2 kept for Sonnet 4.6, learned refusal beats extraBody and model params.openai-temperature-retry.test.ts: refused-temperature retry, unary and streamed. No resend when the refusal is on record and the body carried no temperature. No temperature toclaude-sonnet-5-5on the first send.openai-native-messages.test.ts: reasoning DeepSeek requires back (uncapped, empty field on calls without reasoning, the field kept on answered calls on the first step of a follow-up turn,reasoningBudget).openai-build-body.test.ts:reasoningRequiredputsreasoning_contenton a cloud body, leaves it off without the option, sends less reasoning instead of failing in a tight window (size pinned), and sizes it against a model-paramsmax_tokensand against strict tools, with no capacity throw and reasoning still sent.openai-provider.test.ts: reasoning goes toapi.deepseek.comand to no other host;isDeepSeekHost.cloud-context.test.ts: current-turn reasoning on the outgoing cloud turns, not in the journal or the text prompt, gone once the turn is answered. One path from session throughbuildPromptto the bodyOpenAiProvidersends toapi.deepseek.com: the turn's reasoning on the call, then an empty field on the first step of the next turn.Not confirmed live
temperaturecome from Anthropic's model documentation, and the exact 400 wording is assumed. Neither has been checked against the OpenAI-compatible endpoint with a real key.reasoning_content: ""on its tool-call messages. Not checked that the API accepts it there.model-params.ts; whichever merges second needs a rebase.Not yet checked on the second Mac: lint and tests run there in one batch after approval.