feat(cost): route trivial subtasks to Haiku by default (P5.T5)#408
Conversation
The complexity router (TaskComplexityRouter, wired in coder.py + the session factory) already routes low-complexity subtasks to a cheaper tier, but the 'low' tier defaulted to openai/gpt-4o-mini — which under the default claude-only autonomy is refused and falls back to Sonnet, so the cost saving never fired for the common setup. - config/model_routing.yaml + DEFAULT_MODEL_ROUTING_CONFIG: low tier now claude/claude-haiku-4-5-20251001. Haiku stays inside the Claude runtime (works at AUTO_CODE_AUTONOMY=claude, no second provider) and is far cheaper than the Sonnet fallback. Override to gpt-4o-mini via MODEL_ROUTER_LOW_* env vars or the YAML. - the routed model flows through with_provider_model → the session model, so it's recorded in token_stats.json / cost_report.json (P5.T1/T2). Tests: updated the routing-table + defaults assertions; new test_low_complexity_routes_to_haiku_under_claude_only proves a trivial subtask routes to Haiku with no fallback under a claude-only allowlist; the two fallback/allowlist tests retargeted to the medium (OpenAI) tier so they still exercise provider fallback. 48 passed (task_router + multi_model_orchestration), ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Oleg Miagkov <mrobenner@gmail.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
📝 WalkthroughWalkthroughLow-complexity routing now defaults to Claude claude-haiku-4-5-20251001 in both configuration and routing resolution. The task router adds a cheap-model map for low-tier fallback selection, and tests are updated for the new provider/model expectations and fallback behavior. ChangesLow-complexity routing default change
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Adversarial review caught a regression: with the low tier now defaulting to claude, a non-Claude single-provider setup (e.g. AI_ENGINE_PROVIDER= openai, no Claude creds) would fall back for trivial subtasks to the provider's DEFAULT model (gpt-5.2, $2.50/$10) instead of a cheap one — ~16x the old gpt-4o-mini and inverting the low < medium cost ordering. - CHEAP_MODEL_BY_PROVIDER maps each provider to its Haiku-equivalent (claude→haiku, openai→gpt-4o-mini, google→gemini-1.5-flash) - _resolve_available_route_config: for the low tier, the fallback now uses the cheap model of the available provider, not its default - test: openai-only runtime routes a trivial subtask to gpt-4o-mini (would be the openai default without the fix) 49 passed, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Oleg Miagkov <mrobenner@gmail.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/test_task_router.py (1)
126-140: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winNo test coverage for the
Tests cover claude-only (Haiku direct) and openai-fallback (
gpt-4o-minicheap fallback), but there's no equivalent assertion for aCHEAP_MODEL_BY_PROVIDER["google"]. Given the model ID itself is likely stale (see companion comment ontask_router.py), a test would have caught this drift.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_task_router.py` around lines 126 - 140, Add test coverage in test_task_router for the google cheap-fallback path by exercising a google-only or fallback routing case in the same area that asserts low_route/medium_route/high_route behavior. Verify the route selected by TaskRouter uses CHEAP_MODEL_BY_PROVIDER["google"] for the cheap model choice, similar to how the existing assertions cover claude and openai fallback paths. Reference the existing test setup around low_route, medium_route, high_route, and provider_config so the new assertion fits the current routing matrix.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/backend/core/providers/task_router.py`:
- Around line 24-33: The hardcoded CHEAP_MODEL_BY_PROVIDER map in task_router.py
duplicates the config-driven low-tier model settings and can drift from
model_routing.yaml/task_router_config.py. Update the fallback routing logic in
the task router to derive cheap models from the existing configuration source
used for MODEL_ROUTER_LOW_PROVIDER and MODEL_ROUTER_LOW_MODEL, or add a
config-backed cheap_models section and load it there. Keep the provider fallback
behavior in sync with the config so changes to the low-tier default
automatically apply across all providers.
---
Outside diff comments:
In `@tests/test_task_router.py`:
- Around line 126-140: Add test coverage in test_task_router for the google
cheap-fallback path by exercising a google-only or fallback routing case in the
same area that asserts low_route/medium_route/high_route behavior. Verify the
route selected by TaskRouter uses CHEAP_MODEL_BY_PROVIDER["google"] for the
cheap model choice, similar to how the existing assertions cover claude and
openai fallback paths. Reference the existing test setup around low_route,
medium_route, high_route, and provider_config so the new assertion fits the
current routing matrix.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: c8d53404-0864-4c2b-9091-03a2734c2d26
📒 Files selected for processing (2)
apps/backend/core/providers/task_router.pytests/test_task_router.py
| # Cheap "Haiku-equivalent" model per provider. Used when the low (trivial) | ||
| # tier can't reach its configured provider and must fall back: without this, | ||
| # the fallback would use the provider's *default* (expensive) model, silently | ||
| # upgrading trivial subtasks and inverting the low < medium cost ordering. | ||
| CHEAP_MODEL_BY_PROVIDER: dict[str, str] = { | ||
| "claude": "claude-haiku-4-5-20251001", | ||
| "openai": "gpt-4o-mini", | ||
| "google": "gemini-1.5-flash", | ||
| } | ||
|
|
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy lift
Hardcoded model map duplicates the config-driven default and can silently drift.
CHEAP_MODEL_BY_PROVIDER is a second, code-level source of truth for "the cheap model per provider," parallel to model_routing.yaml/task_router_config.py which already define the low-tier default (claude/claude-haiku-4-5-20251001) and its overrides (MODEL_ROUTER_LOW_PROVIDER/MODEL_ROUTER_LOW_MODEL). If the config-driven low-tier default is changed later (e.g., via env var or YAML), this hardcoded map won't follow, so a claude → openai fallback could route to a stale gpt-4o-mini while the "primary" low tier has moved on. Consider deriving per-provider cheap models from configuration (e.g., a cheap_models section in model_routing.yaml) rather than a separate Python constant.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/backend/core/providers/task_router.py` around lines 24 - 33, The
hardcoded CHEAP_MODEL_BY_PROVIDER map in task_router.py duplicates the
config-driven low-tier model settings and can drift from
model_routing.yaml/task_router_config.py. Update the fallback routing logic in
the task router to derive cheap models from the existing configuration source
used for MODEL_ROUTER_LOW_PROVIDER and MODEL_ROUTER_LOW_MODEL, or add a
config-backed cheap_models section and load it there. Keep the provider fallback
behavior in sync with the config so changes to the low-tier default
automatically apply across all providers.
1 participant
What
Roadmap P5·T5: «Роутинг дешёвой модели на тривиальные подзадачи … Простые подзадачи идут на haiku, записано в статистике».
The complexity router (
TaskComplexityRouter) is already fully wired — it's called per-subtask inagents/coder.py(the live build loop) and in the shared session factory (core/providers/factory.py), routing low-complexity subtasks to a cheaper tier. But thelowtier defaulted toopenai/gpt-4o-mini, which under the default claude-only autonomy (AUTO_CODE_AUTONOMY=claude, allowlist{claude,codex}) is refused and falls back to Sonnet — so the cost saving never fired for the common Claude-only setup.lowtier is nowclaude / claude-haiku-4-5-20251001. Haiku stays inside the Claude runtime (no second provider needed) and is far cheaper than the Sonnet fallback claude-only users get today. Override to gpt-4o-mini viaMODEL_ROUTER_LOW_PROVIDER/MODEL_ROUTER_LOW_MODELor the YAML.route.model → with_provider_model → session model → save_token_stats, so it lands intoken_stats.json/cost_report.json(P5·T1/T2). No extra wiring needed.Validation passes:
claude-haiku-4-5-20251001is incost_calculator.MODEL_PRICINGand theclaude→anthropicpricing alias is accepted. Cost ordering high(Sonnet 0.045) > medium(gpt-4o 0.0325) > low(Haiku 0.012) still holds.Acceptance
A trivial subtask under a claude-only runtime now routes to Haiku (not the Sonnet fallback), proven by
test_low_complexity_routes_to_haiku_under_claude_only.Tests
test_low_complexity_routes_to_haiku_under_claude_only; the two fallback/allowlist tests retargeted to themedium(OpenAI) tier so they still exercise provider fallback.test_task_router.py+test_multi_model_orchestration.py), ruff check + format clean.Roadmap doc line deferred to avoid a merge conflict with #407's in-flight roadmap edit.
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes
Tests