Conversation
Send each classifier stage the output room its context-fit check already reserves: the answer allowance plus the reasoning budget. On providers where hidden reasoning shares the output ceiling, reasoning no longer consumes the decision tool call room. After a length stop, the detailed retry now re-requests with the largest context-safe ceiling (model output limit capped to context room) instead of repeating the identical request. Attempts record requestMaxTokens. Refs czottmann#42, czottmann#39
The composed maxTokens matches the czottmann#51 reserve. It does not partition hidden reasoning from the answer, and the length retry is not context-safe when the base ceiling already exceeds the window.
When the model output limit already bounds the request (at xhigh and max, or on models whose output limit is at most the allowance plus the reserve), the escalated retry ceiling equals the first ceiling. The retry would repeat the identical request at up to the model output limit. Fail closed after the first length stop instead. Also document the fast-stage composed ceiling at xhigh and max, mark the ceiling raise as explicit-level only, and derive the 33,968 and 33,280 figures from the allowance plus the reserve.
- flow.md: drop the "the retry keeps it" sentence; the skip falsifies it (the new staged-path test asserts a single call in that case). - classifier.ts: the skip comment named only the model output limit as the bounding cause; the estimated context room also bounds direct classifyWithRetry callers. - configuration.md: qualify the fastClassifierMaxTokens advice; raising the allowance does not change the sent ceiling once the model output limit is at most the allowance plus the reserve. - CHANGELOG: drop the (czottmann#42) marker; this change does not fix czottmann#42. - tests: classifyInStages skip coverage at explicit xhigh with the composed ceiling clamped to model.maxTokens (verified red at the assertion level against c372f62).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #42
Refs #39
What changed
Both classifier stages now pass
min(answer allowance + #51 reasoning reserve, model.maxTokens)asmaxTokens. At explicitlowthat is 5296 for the detailed stage (1200 + 4096) and 4608 for the fast stage (512 + 4096). With no reasoning level set, the requests stay at 1200 and 512.The #51 reserve is the OMP 18 table already used by the context-fit check (
low4096,xhigh/max32768). It is not pi-ai 0.86.0's thinking-pad table (low2048;xhighandmaxclamp tohigh16384). Sending the reserve keeps the completion-function argument equal to the amount the fit check reserved. It does not make that amount the provider's reasoning budget.On a shared-ceiling provider the composed value is one output cap. Hidden reasoning can still consume it, including the room needed for
classifier_decision. On a budget-thinking provider the adapter may add its own thinking budget on top of this value and then clamp to the context window, so the wire limit can exceed the reserve. Atxhighandmax, a composed ceiling is the model output limit whenever that limit is at most the allowance plus the reserve (33,968 for the detailed stage, 33,280 for the fast stage).A detailed-stage
lengthstop used to retry the same request. It now retries once at the smaller ofmodel.maxTokensandcontextWindow - estimatedInput - 4096, and never below the first ceiling. When the escalated ceiling cannot exceed the first ceiling, no retry is sent; the action fails closed after the firstlengthstop. On the staged path this happens when the model output limit already bounds the request; directclassifyWithRetrycallers can also reach it through the estimated context room. A secondlengthstop still fails closed. Malformed output still retries at the same ceiling. Timeout and provider errors still fail closed without a retry.This can be a large retry. A model whose output limit is 384000 can be asked for 384000 tokens on the second attempt.
classifierTimeoutMsbounds time, not that token reservation. Codex responses ignoremaxTokens, so neither the composed ceiling nor the retry cap applies there.ClassifierIoAttemptrecordsrequestMaxTokens. CHANGELOG and the classifier docs describe the ceiling as a raised cap, not as a partition between reasoning and the answer.Why this shape
The fit check already reserved
allowance + #51 reserveand the request sent only the allowance. Those two numbers now match at the completion-function boundary. That is an internal consistency fix. It is not evidence that a shared-ceiling model will finish inside the new cap. Whether the #39 truncations fit in 5296 was not measured.There is no new configuration. #39 asks for a
classifierDetailedMaxTokensknob. This PR does not add one.fastClassifierMaxTokensstill changes the fast-stage allowance; it raises the sent ceiling only while the model output limit exceeds the allowance plus the reserve.This replaces #43. That PR was built on 4d902a2, before the token estimator and the #51 reserves. It also sent
model.maxTokenson the first attempt. This branch sends that only when the composed ceiling clamps to the model limit, or on a length retry.Commit-message note: the first commit on this branch (
daf6c36) says that on shared-ceiling providers reasoning "no longer consumes the decision tool call room". That claim is withdrawn; it predates the analysis above and is superseded by it. History is left intact.Tests
The new tests pin the #51 reserve table, the composed ceilings, and the retry arguments, including the case where a base ceiling of 32000 is returned unchanged for a 10000-token window, and the skipped retry when the ceiling cannot rise (the model output limit equal to the base, and a base above the estimated context room). A staged-path test covers the production wiring: explicit
xhighwith the composed ceiling clamped tomodel.maxTokensfails closed after a single detailed call on alengthstop, verified red at the assertion level against the pre-skip source. They do not show that 5296 is enough for a shared-ceiling model. The first red check against unmodified source was an import failure from the new exports, not an assertion failure; the skip tests were verified red at the assertion level before the implementation. Full suite: 329 tests, 328 pass, one pre-existing skip.npm run checkis clean.