Skip to content

feat: allow classifierReasoningLevel "off" to disable classifier thinking - #58

Open
bsamek wants to merge 1 commit into
czottmann:mainfrom
bsamek:allow-classifier-reasoning-off
Open

bsamek wants to merge 1 commit into
czottmann:mainfrom
bsamek:allow-classifier-reasoning-off

Conversation

@bsamek

@bsamek bsamek commented Sep 21, 2026

Copy link
Copy Markdown

classifierReasoningLevel: "off" is rejected by validation, but the runtime already supports it

Summary

AutoModeSettings.classifierReasoningLevel only accepts low | medium | high | xhigh | max, so there is no way to ask the classifier to disable thinking. Meanwhile, createClassifierCompletionPlan() in classifier.ts already has a complete effectiveLevel === "off" execution path (it selects the simple completion and omits the reasoning option) — it's just unreachable via config, because the config validator excludes "off".

Why "off" matters

Several classifier-worthy models support a true non-thinking mode via Pi AI's thinkingLevelMap:

Model thinkingLevelMap.off Effect when no reasoning option is sent
fireworks/.../deepseek-v4p1-flash "none" anthropic-messages API sends thinking: { type: "disabled" }
fireworks/.../glm-5p2 "none" openai-completions sends reasoning_effort: "none"
fireworks/.../glm-5p3 null nothing sent; server default (thinking on)
github-copilot/gpt-5.6-luna null clamps up to minimal (→ low effort)

With no reasoning preference (the current behavior when classifierReasoningLevel is absent), these "off-capable" models still run their server-default reasoning on every classification. Requesting "off" explicitly makes the fast stage genuinely non-thinking — a large latency win for a gate that runs on every non-read-only tool call.

Proposed change

Accept "off" in ClassifierReasoningLevel and in config validation. The existing clamping semantics already handle models that cannot disable thinking: clampThinkingLevel(model, "off") walks up to the nearest supported level (e.g. minimal for GLM-5.3 / GPT-5.6 Luna), so behavior degrades gracefully instead of failing.

Full diff attached as 0001-allow-classifierReasoningLevel-off.patch. It touches:

  • extensions/auto-mode/types.ts — add "off" to ClassifierReasoningLevel; narrow EffectiveClassifierReasoningLevel so the union shape is unchanged (all Exclude<EffectiveClassifierReasoningLevel, "off"> sites are unaffected).
  • extensions/auto-mode/config.ts — add "off" to CLASSIFIER_REASONING_LEVELS; update the diagnostic message.
  • docs/configuration.md — document "off" and the clamping behavior.

Verification

Tested locally against v1.16.0 with Pi AI's real model metadata:

  • isClassifierReasoningLevel("off") === true; garbage values still rejected.
  • createClassifierCompletionPlan(deepseek-v4p1-flash, "off", …) → { effectiveLevel: "off" }, simple-completion path, no reasoning option sent → pi-ai emits thinking: { type: "disabled" } on the wire.
  • createClassifierCompletionPlan(glm-5p3, "off", …) → clamps to minimal (same as clampThinkingLevel semantics used for other levels).
  • End-to-end: ran pi with classifierModel: fireworks/accounts/fireworks/models/deepseek-v4p1-flash and classifierReasoningLevel: "off"; classifier decisions (fast + detailed stages) round-tripped successfully with no reasoning content in responses.

Suggested test

test("classifierReasoningLevel off resolves to the off path for off-capable models and clamps otherwise", async () => {
  const raw = makeStubCompletion("raw");
  const simple = makeStubCompletion("simple");
  const offCapable = { id: "m", provider: "fireworks", reasoning: true, thinkingLevelMap: { off: "none", low: "low", high: "high" } };
  const plan = createClassifierCompletionPlan(offCapable, "off", raw, simple);
  assert.equal(plan.completeFn, simple);
  assert.deepEqual(plan.reasoning, { mode: "explicit", requestedLevel: "off", effectiveLevel: "off" });
  assert.equal(plan.reasoningLevel, undefined);

  const thinkingLocked = { id: "m", provider: "fireworks", reasoning: true, thinkingLevelMap: { off: null, low: "low", high: "high" } };
  const clamped = createClassifierCompletionPlan(thinkingLocked, "off", raw, simple);
  assert.equal(clamped.reasoning.effectiveLevel, "minimal");
});

…king

Accept "off" in classifierReasoningLevel so the classifier can run
without thinking on models that support it (per pi-ai's
thinkingLevelMap, e.g. DeepSeek V4.1 Flash and GLM-5.2). The runtime
already had a complete effectiveLevel === "off" path in
createClassifierCompletionPlan(); it was unreachable via config because
validation excluded the value.

Models that cannot disable thinking keep the existing clamping
semantics: clampThinkingLevel resolves "off" to the nearest supported
level (e.g. minimal for GLM-5.3 / GPT-5.6 Luna).

Verified end-to-end against Fireworks deepseek-v4p1-flash: classifier
requests send no reasoning option, pi-ai emits thinking disabled on the
wire, responses contain zero reasoning tokens, and decisions parse on
the first attempt.
@bsamek
bsamek marked this pull request as ready for review September 21, 2026 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant