Skip to content

fix(classifier): bound detailed stage by time, not a fixed ceiling - #43

Closed
ArietidsZ wants to merge 1 commit into
czottmann:mainfrom
ArietidsZ:fix/42-detailed-stage-time-bound
Closed

ArietidsZ wants to merge 1 commit into
czottmann:mainfrom
ArietidsZ:fix/42-detailed-stage-time-bound

Conversation

@ArietidsZ

Copy link
Copy Markdown

Closes #42.

What

The detailed classifier stage's output ceiling was hardcoded at 1200 tokens and always applied. On providers that count reasoning tokens against the completion ceiling (OpenAI-compatible, OpenAI Responses), a reasoning classifier could spend the ceiling before emitting decision JSON, stop on length, truncate again on the single retry, and block the action.

The detailed stage now sends the classifier model's own output limit, capped to the room left in the context window. classifierTimeoutMs stays the per-request bound for both stages. No new config key.

Changes

  • DETAILED_CLASSIFIER_MAX_TOKENS becomes DETAILED_ANSWER_ALLOWANCE_TOKENS, used only by the context-fit reserve (min(answerAllowance + reasoningBudget, modelMaxTokens)).
  • classifyWithRetry defaults maxTokens to classifier.model.maxTokens.
  • New classifierDetailedMaxTokens() caps the detailed ceiling to contextWindow - 4096 - fixedInputUpperBound - actionBytes. This matters because Pi's raw provider stream (the default path when classifierReasoningLevel is unset) forwards maxTokens without Pi's own context clamp, so a bare model limit can exceed the window on models where maxTokens >= contextWindow (~13% of the shipped catalog). The cap keeps the byte-conservative input bound plus the ceiling inside the window, and guarantees at least answerAllowance + reasoningBudget of output room whenever the fit check passes.
  • defaultClassifyAction computes the ceiling and passes it through StagedClassifierOptions.detailedMaxTokens.
  • Docs and CHANGELOG updated.

Behavior

  • Fail-closed behavior unchanged: timeout, provider error, non-stop, malformed JSON, and a missing model still block. length still retries once.
  • Fast stage unchanged (fastClassifierMaxTokens, default 512).
  • Config unchanged: no new keys.

Tests

npm run check and npm test. New tests cover the detailed request ceiling, the context-window cap, the model-limit binding case, and the reserve bound.

Send the detailed request the classifier model's own output limit, capped to
the room left in the context window, instead of the hardcoded 1200-token
ceiling. classifierTimeoutMs stays the per-request bound for both stages.

Providers that count reasoning tokens against the completion ceiling no
longer truncate before the decision JSON. The context-fit reserve keeps its
conservative answer allowance, bounded by the model output limit. No new
config key.
@ArietidsZ

ArietidsZ commented Sep 22, 2026 •

Copy link
Copy Markdown
Author

Closing this in favor of #61.

This PR was built against 4d902a2, before the classifier overhaul, so it no longer applies to main: #37 replaced the byte-based cap it computed with a token estimator, and #51 added reasoning reserves to the context checks.

Correction to an earlier version of this comment: #61 does not implement this proposal, and the earlier wording ("implements the same goal") was wrong. This PR sends the model output limit on the first request and makes the per-request timeout the operative bound. #61 keeps a fixed composed ceiling (answer allowance plus the #51 reserve) on the first attempt and escalates only after a length stop, to min(model.maxTokens, estimated context room), skipping the retry when the ceiling cannot rise. The two share the constraints — no new configuration, fail-closed, classifierTimeoutMs still enforced — but not the mechanism: #61's first attempt still sends a fixed ceiling rather than the model output limit, so it does not deliver this proposal's attempt-1 coverage.

@ArietidsZ ArietidsZ closed this Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bound the detailed classifier stage by time instead of the fixed 1200-token ceiling

1 participant