Allow agents to use a small/fast model for lightweight turns
Summary
Long-running multi-step tasks are slow because every agent turn uses the session's main model — including lightweight steps (status updates, confirmations, plain-text continuations). The Catalog.model.small() helper already exists and is used for title generation, but agent turns never use it.
Proposal
Add an opt-in small flag to AgentV2.Info. When an agent sets small: true, the runner resolves a small model via the existing Catalog.model.small(providerID) and prefers it for the turn, falling back to the session model when no small model is available.
- Zero behavior change for existing agents (flag defaults to absent)
- Never fails a turn when a small model is missing
- Reuses the existing
Catalog.model.small() (keyword + cost/age scoring)
Motivation
This reduces per-turn latency and token cost for agents that spend most steps on non-critical work, which is the dominant cost in long autonomous runs.
Related
Allow agents to use a small/fast model for lightweight turns
Summary
Long-running multi-step tasks are slow because every agent turn uses the session's main model — including lightweight steps (status updates, confirmations, plain-text continuations). The
Catalog.model.small()helper already exists and is used for title generation, but agent turns never use it.Proposal
Add an opt-in
smallflag toAgentV2.Info. When an agent setssmall: true, the runner resolves a small model via the existingCatalog.model.small(providerID)and prefers it for the turn, falling back to the session model when no small model is available.Catalog.model.small()(keyword + cost/age scoring)Motivation
This reduces per-turn latency and token cost for agents that spend most steps on non-critical work, which is the dominant cost in long autonomous runs.
Related