Skip to content

feat(core): allow agents to opt into a small model for lightweight turns - #46928

Open
everest-an wants to merge 1 commit into
anomalyco:devfrom
everest-an:agent-small-model
Open

feat(core): allow agents to opt into a small model for lightweight turns#46928
everest-an wants to merge 1 commit into
anomalyco:devfrom
everest-an:agent-small-model

Conversation

@everest-an

Copy link
Copy Markdown

Allow agents to use a small/fast model for lightweight turns

Problem

The SessionRunner resolves a single model per session (SessionRunnerModel.resolve) and uses it for every provider turn. The Catalog.model.small() helper already exists and is used for title generation, but agent turns never take advantage of it — so lightweight steps (status updates, confirmations, plain-text continuations) pay the full cost of the main model, which slows down long-running multi-step tasks.

Proposal

Add an opt-in small flag to the agent schema (AgentV2.Info). When an agent sets small: true, the runner resolves a small model via the existing Catalog.model.small(providerID) and prefers it for the turn, falling back to the session model when no small model is available.

  • Zero behavior change for existing agents (flag defaults to absent).
  • Never fails a turn when a small model is missing.
  • Reuses the existing Catalog.model.small() (keyword + cost/age scoring) rather than inventing new model-selection logic.

Changes

  • packages/schema/src/agent.ts: add optional small boolean to AgentV2.Info
  • packages/core/src/session/runner/model.ts: add SessionRunnerModel.resolveSmall
  • packages/core/src/session/runner/llm.ts: prefer the small model when agent.info?.small is set
  • packages/core/test/session-runner-model.test.ts: unit tests for resolveSmall injection + fallback

Notes

  • This is a first slice; a follow-up could add automatic "lightweight turn" detection instead of requiring the explicit flag.

@github-actions github-actions Bot added the needs:compliance This means the issue will auto-close after 2 hours. label Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

This PR doesn't fully meet our contributing guidelines and PR template.

What needs to be fixed:

  • PR description is missing required template sections. Please use the PR template.

Please edit this PR description to address the above within 2 hours, or it will be automatically closed.

If you believe this was flagged incorrectly, please let a maintainer know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs:compliance This means the issue will auto-close after 2 hours.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant