Skip to content

agents: enforce aggregate AI usage limits - #511

Open
demattosanthony wants to merge 1 commit into
ai-limits/02-storage-providersfrom
ai-limits/03-enforcement
Open

agents: enforce aggregate AI usage limits#511
demattosanthony wants to merge 1 commit into
ai-limits/02-storage-providersfrom
ai-limits/03-enforcement

Conversation

@demattosanthony

@demattosanthony demattosanthony commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Enforce aggregate AI usage limits across direct Agent conversations and workflow Agent nodes.

This PR adds a fast request-time preflight and an authoritative atomic reservation immediately
before every provider call. Completed calls record usage, valuation, and reservation reconciliation
in one storage transaction. Calls that may have been billed without trustworthy usage retain their
reservation as unknown capacity instead of failing open.

Enforcement model

The request preflight rejects obviously exhausted or unsafe runs before creating a new conversation
thread or message. It is an early user-experience optimization, not the concurrency authority.

The worker boundary remains authoritative:

  1. Build the exact prepared provider request after context compaction.
  2. Generate a stable Sixb-owned call ID.
  3. Estimate text input tokens and a conservative internal output allowance.
  4. Pre-price the call against the pinned Models.dev catalog when a cost policy applies.
  5. Atomically reserve every applicable project, group, and requester bucket.
  6. Invoke the provider only after reservation succeeds.
  7. Record immutable usage and valuation, then reconcile the reservation in the same transaction.

The same recorder and admission controller are used by conversational turns and structured workflow
Agent nodes.

What changes

  • Add ai.usage_limit_exceeded and ai.usage_limit_unavailable as stable coded failures.
  • Document both new catalog codes, their boundary membership, and their recovery guidance alongside
    the catalog change.
  • Add preflight to new and retried Agent-run requests before mutable conversation state is created.
  • Add provider-neutral request estimation and reservation inputs.
  • Add conservative catalog cost estimation using the existing pinned Models.dev snapshot.
  • Extend model middleware with awaited admission and stable provider-attempt identity.
  • Integrate reservation reconciliation with synchronous accounting and durable recovery.
  • Mark potentially billed calls unknown when provider or callback completion is ambiguous.
  • Enforce limits in the run-owned conversational runtime and workflow Agent-node execution.
  • Preserve limit failures in Agent and workflow failure unions instead of collapsing them into
    generic execution failures.
  • Map direct HTTP rejection to 429, expose safe structured details, and set Retry-After from the
    earliest applicable reset.
  • Regenerate public client artifacts affected by failure-code and Agent-route schema changes.

Failure and safety behavior

  • Exhaustion is non-retryable until capacity or policy state changes.
  • Unsafe evaluation is retryable after storage or accounting health is restored.
  • An applicable cost policy fails closed when model identity or pricing dimensions cannot be
    estimated safely.
  • Incomplete historical accounting fails closed for the affected meter.
  • A later denied model step preserves already-streamed coherent output and already-recorded usage.
  • Sixb reservations bound its own admission decisions; they do not replace provider-side invoice
    budgets or guarantee that one response cannot exceed its conservative reservation.

Reviewer guide

Suggested review order:

  1. model-call-admission.ts and model-call-limits.ts for estimation and policy selection.
  2. models-dev/pricing.ts for conservative pre-call cost bounds.
  3. model-call-recorder.ts, accounting, and recovery for the reservation lifecycle.
  4. Conversation and workflow integration points.
  5. Stable failures, error-code documentation, and HTTP 429 mapping.
  6. Tests covering rejection-before-provider, concurrent overrun, recovery, and partial output.

Generated client files should be reviewed as derived output; the source contracts are the Agent
failure unions and route response schemas.

Validation performed

  • All ten changed core, Agent-worker, and server test files — 201 passed.
  • bun run generate:client — passed; derived failure contracts were regenerated from source.
  • bun run typecheck — passed on this branch.
  • bun run check — passed on this branch.
  • The complete stack tip also passed idempotent client generation, the full monorepo build,
    publish-boundary validation, and the guarded unit suite.

Not in this PR

  • Policy-management or status HTTP endpoints.
  • AI usage observation or management grants.
  • Atlas management UI.
  • User-configurable per-call output ceilings, execution-root budgets, or provider invoice
    reconciliation.

Stack

This is PR 3 of 5 in GitHub stack #514. It depends on
#510. Next:
#512, which exposes the protected operator API after
enforcement is authoritative.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant