Conversation
Send the detailed request the classifier model's own output limit, capped to the room left in the context window, instead of the hardcoded 1200-token ceiling. classifierTimeoutMs stays the per-request bound for both stages. Providers that count reasoning tokens against the completion ceiling no longer truncate before the decision JSON. The context-fit reserve keeps its conservative answer allowance, bounded by the model output limit. No new config key.
|
Closing this in favor of #61. This PR was built against 4d902a2, before the classifier overhaul, so it no longer applies to main: #37 replaced the byte-based cap it computed with a token estimator, and #51 added reasoning reserves to the context checks. Correction to an earlier version of this comment: #61 does not implement this proposal, and the earlier wording ("implements the same goal") was wrong. This PR sends the model output limit on the first request and makes the per-request timeout the operative bound. #61 keeps a fixed composed ceiling (answer allowance plus the #51 reserve) on the first attempt and escalates only after a |
Closes #42.
What
The detailed classifier stage's output ceiling was hardcoded at 1200 tokens and always applied. On providers that count reasoning tokens against the completion ceiling (OpenAI-compatible, OpenAI Responses), a reasoning classifier could spend the ceiling before emitting decision JSON, stop on
length, truncate again on the single retry, and block the action.The detailed stage now sends the classifier model's own output limit, capped to the room left in the context window.
classifierTimeoutMsstays the per-request bound for both stages. No new config key.Changes
DETAILED_CLASSIFIER_MAX_TOKENSbecomesDETAILED_ANSWER_ALLOWANCE_TOKENS, used only by the context-fit reserve (min(answerAllowance + reasoningBudget, modelMaxTokens)).classifyWithRetrydefaultsmaxTokenstoclassifier.model.maxTokens.classifierDetailedMaxTokens()caps the detailed ceiling tocontextWindow - 4096 - fixedInputUpperBound - actionBytes. This matters because Pi's raw provider stream (the default path whenclassifierReasoningLevelis unset) forwardsmaxTokenswithout Pi's own context clamp, so a bare model limit can exceed the window on models wheremaxTokens >= contextWindow(~13% of the shipped catalog). The cap keeps the byte-conservative input bound plus the ceiling inside the window, and guarantees at leastanswerAllowance + reasoningBudgetof output room whenever the fit check passes.defaultClassifyActioncomputes the ceiling and passes it throughStagedClassifierOptions.detailedMaxTokens.Behavior
stop, malformed JSON, and a missing model still block.lengthstill retries once.fastClassifierMaxTokens, default 512).Tests
npm run checkandnpm test. New tests cover the detailed request ceiling, the context-window cap, the model-limit binding case, and the reserve bound.