Skip to content

Better output-token prediction for pre_call_worst_case #29

Description

@susheem-k

Summary

Worst-case gating depends on max_output / defaults (e.g. 1024). Uncapped calls MUTATE to a default; projection is coarse. Bad estimates → false HALTs or weak prevention.

Area: pre_call_worst_case, CallRequest.max_output_tokens, est_input

Desired behavior

  • Better estimators: task/intent priors, model context window, historical p50/p95 for this agent+intent, optional caller hint.
  • Keep fail-closed on unknown price; never price physical model max without a bound.
  • Document accuracy limits; prefer MUTATE-to-priced-cap over crude HALT when headroom exists.

Acceptance sketch

  • Measurable reduction in false-positive pre_call HALTs on demo workloads without raising overspend rate.

Reference: docs/notes/2026-07-23-design-fix-notes.md §13

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions