Repository navigation
docs(dev): v1.0 design draft — cross-agent session budget cap - #34
Conversation
…atus DESIGN-v1.0-budget-cap.md: a signed budget.yaml (principal needs the new budget class) with per-session and per-project-per-day caps in tokens or estimated USD, measured from each agent's local session log incrementally, enforced by the existing hook on every tool call (and new prompts where the agent allows it): warn once at a threshold, deny at 100%. Built-in price table with signed overrides; unknown log formats fail open with a warning by default. Threats and limits stated (an agent can rewrite its own log: a runaway guard, not an adversarial control). Also defines what 1.0 makes stable. PLAN item 11 and the TODO point to it; PLAN gains a 2026-10-07 status section.
moneytool
left a comment
There was a problem hiding this comment.
The design needs the following clarifications before merging. All current CI checks pass; these findings concern the proposed guarantees and accounting behavior.
| - **Below `warn_at`:** nothing changes; budget adds no output. | ||
| - **Crossing `warn_at`:** one warning per session per threshold (not per call), with usage, the | ||
| limit and how much is left. | ||
| - **At 100%:** every tool call is denied, and every new prompt where possible, with one message: |
There was a problem hiding this comment.
[P1] Clarify the spending guarantee. The proposed tool hooks read usage after model requests have already incurred cost. Denying the next tool call cannot prevent that request from crossing the limit, and parallel requests or model responses without tool calls can continue spending beyond the cap. Please describe the guarantee as stopping subsequent tool activity once observed usage reaches the threshold, explicitly document overshoot and tool-free activity, or specify enforcement before each model request if a hard spending ceiling is required. Align the hard-stop and ceiling claims elsewhere in the draft with that guarantee.
There was a problem hiding this comment.
Agreed, fixed in 619d714. The guarantee is now stated as stopping subsequent tool activity (and new prompts where a prompt hook exists) once observed usage reaches the limit. New §6.1 lists the overshoot: the request that crossed the limit, the reply to the denial, in-flight parallel sub-agents and sessions, and tool-free chat where there is no prompt hook. It gives the bound and says a hard ceiling needs enforcement before each model request (a gateway, or --max-budget-usd). That is now a v1.0 non-goal in §2 and open question 7. §1, §2, §6 and the threats table use the same wording, and aegis budget status reports usage over the limit as overshoot.
There was a problem hiding this comment.
Thanks, the narrower guarantee in 619d714 addresses the original hard-ceiling concern. To finish this, remove the remaining claims that overshoot is bounded in §2 and the threats table, and replace the rough one-turn bound in §6.1 with illustrative sources of overshoot. Denying a tool does not establish that the agent makes only one subsequent model request; it may retry, and tool-free activity is already described as unbounded. Suggested wording: 'Aegis denies covered tool calls and prompts when the usage available at that hook reaches the limit. It does not impose a finite bound on additional model spending; in-flight requests, delayed usage records, retries after denial, and tool-free activity can add cost.' Keep status reporting of observed overshoot. Validate the implementation with delayed usage records, parallel sessions, repeated denied-tool retries, and turns without tools. Treat gateways and vendor limits as alternative enforcement layers whose guarantees must be verified, rather than promising that they necessarily provide a strict ceiling.
There was a problem hiding this comment.
Done in 77fdf49.
- §6.1 now uses your guarantee wording verbatim.
- The rough one-turn bound is gone. In its place is an illustrative list: in-flight requests, delayed usage records (for example Copilot CLI writes tokens only at shutdown), retries after denial (any number of turns), and tool-free activity.
- §1, the §2 goal and non-goal, and the threats row now say spend past the limit is not bounded. Observed overshoot is still reported by
aegis budget status. - Gateways, vendor limits and
--max-budget-usdare described as alternative enforcement layers whose guarantees must be verified for each deployment. - The validation plan (§10 step 5) adds delayed records, parallel sessions, repeated denied-tool retries and turns without tools.
| offset (or the last row id for SQLite), running totals per model, last update time. A file | ||
| that shrank, or an offset past its end, means the totals are recomputed from the start. The | ||
| state is a cache: deleting it costs a re-read, never resets the budget (§8). | ||
| - **Project/day totals** are the sum, over sessions whose working directory is inside the |
There was a problem hiding this comment.
[P2] Define project-total reconstruction and accounting. The draft promises that deleting cached state never resets the budget, but it does not define how all relevant sessions are discovered, how subagents are attributed to a project, or how concurrent hooks update per-session offsets and project totals without counting the same usage twice. A lock on the project file alone does not define that transaction. Please specify session discovery, project/subagent attribution, an idempotent update protocol, and reconstruction after either session or project cache deletion, including what happens when required logs are unavailable.
There was a problem hiding this comment.
Fixed in 619d714, new §4.1. In short: the logs are the only source of truth. Per-session records hold the offset, file identity and day/model buckets. The project map is keyed by (agent, session), and each update replaces that session's entry, so the project total is a sum computed on read, never a running counter. Protocol: take the session lock, re-read the record, check the file identity, read to the last complete record, write atomically, then take the project lock and set the keyed entry. Lock order is always session → project, so a racing hook reads nothing new and a crash between steps self-heals. Project attribution uses the starting cwd from the log (innermost .aegis/ wins). A sub-agent counts toward its parent session where the log links them (Claude Code <session>/subagents/), otherwise as its own session in the project. A per-parser discover(project, day) rebuilds after session, project or full cache deletion. Unavailable logs make the rebuilt total an explicit lower bound and fall under on_unknown_log.
There was a problem hiding this comment.
The keyed replacement approach in 619d714 is a good fix for repeated updates. Please make the rebuild protocol equally explicit before calling this resolved. (1) A rebuild performed under the project lock must not acquire session locks: that would reverse the session -> project order used by normal hooks and can deadlock. Discover/rebuild session records before taking the project lock, then merge their contributions under that lock using monotonic per-session revisions so an older rebuild cannot overwrite a newer hook update. Do not replace the entire project map from an earlier snapshot. (2) Define one contribution owner for linked subagents: their usage is included in the parent contribution and must not also appear as separate project entries. (3) With the whole cache and some logs gone, there is no surviving inventory from which to count missing sessions. Report completeness as unknown rather than reporting zero missing sessions; define strict recovery, such as a durable session inventory outside the disposable cache or denying an unverifiable rebuild until an explicit reset/day boundary. Also do not bucket timestamp-less records by the day of rereading, since cache reconstruction would move old usage into a new day; use stable recorded metadata or classify attribution as unknown. Add crash recovery, concurrent rebuild/update, linked-subagent discovery, missing-history, and midnight reconstruction cases to the implementation validation plan.
There was a problem hiding this comment.
Done in 77fdf49 (§4.1).
- Rebuild without lock inversion. A rebuild lists sessions without the project lock. It updates each session under that session's lock alone, then merges under the project lock. No code path takes a session lock while holding the project lock. Session records carry a monotonic revision; the project map stores
(revision, usage), and every writer merges entry by entry, keeping the newer revision. Nobody writes a whole map from an earlier snapshot. - One owner for linked sub-agents. Their logs are read into the parent's record, so they appear only inside the parent's entry. Their hooks update the parent, and
discoverskips them as top-level sessions. An unlinked sub-agent is its own owner. - Completeness. There is a durable session inventory under
$XDG_STATE_HOME, outside the disposable cache: one append-only line per session plus a checkpoint at most once a minute. A rebuild is reported as complete, partial ("M of N from checkpoints") or unknown (inventory missing); it never reports "0 missing". Partial or unknown is treated as an unknown log. Strict mode denies until the day boundary or anaegis budget reset --daysigned with the policy key. - Timestamp-less records. These use stable recorded metadata: the nearest earlier timestamp in the log, else the session start. With neither, the day is unknown: it counts toward the session cap, stays out of every project/day total, and falls under
on_unknown_log. Usage is never bucketed by the day it is read.
The validation plan (§10 step 5) adds crash recovery, a rebuild concurrent with hook updates, linked sub-agent discovery, missing history (partial and unknown) and midnight reconstruction.
| - A **built-in price table** (`aegis_core/budget/prices.yaml`), versioned and dated, per model: | ||
| input, output, cache-read and cache-write prices per million tokens. It ships with each | ||
| release; `budget.yaml` may override entries (signed, so an agent cannot cheapen its model). | ||
| - **Unknown model:** counted at the most expensive known model of the same vendor, with a |
There was a problem hiding this comment.
[P2] Correct the unknown-model pricing guarantee. The most expensive known model from a vendor is not necessarily an upper bound for a newly introduced model, so this fallback cannot guarantee that an unpriced model never makes a budget go further. The behavior is also undefined when the vendor cannot be identified or has no table entries. Please describe the fallback as an estimate, define unknown-vendor handling, and specify whether strict USD enforcement requires explicit pricing or denies requests whose price cannot be established.
There was a problem hiding this comment.
Fixed in 619d714 (§5, plus a config line and a threats row). The fallback is now described as an estimate that may be low, not an upper bound. Vendor identification is defined (the log's provider field, or the model-id prefix in the table's vendor list). An unknown vendor, or one with no entries, falls back to the most expensive entry in the whole table. A new on_unknown_price: estimate | deny setting covers USD mode: with deny, a session that used a model with no explicit price has its tool calls denied until a pricing: entry is signed. It is ignored in token mode. The default is open question 6.
There was a problem hiding this comment.
The changes in 619d714 address this finding: both vendor and global fallbacks are explicitly estimates, and on_unknown_price: deny gives a defined response to missing prices. I recommend keeping estimate as the general default and documenting deny for users who prefer stopping covered activity when pricing is unknown. Two implementation details to pin down: choose fallback rates conservatively per billing category (input, output, cache read, cache write), since no single model necessarily has the highest rate in every category; and retain usage buckets keyed by the original model id so adding a signed price override can recompute prior estimated usage without double counting. Replace 'exact' in the open question with 'requires explicit prices': known rates alone do not guarantee an exact invoice. Validate unknown vendor, missing price category, explicit override, and token-mode behavior. No further blocker on the original unknown-model guarantee once that wording is aligned.
There was a problem hiding this comment.
Done in 77fdf49.
- Fallback rates are chosen per billing category: the highest input, output, cache-read and cache-write rate taken separately among the vendor's entries, or across the whole table for an unknown vendor. A known model with a missing category gets the fallback for that category only.
- Repricing: usage is stored as tokens per category under the original model id, and dollars are computed when read. A signed override therefore re-prices earlier estimated usage without double counting.
- Defaults:
estimateis the default (Q6, decided).denyis documented for users who prefer to stop covered activity, and the wording says it "requires explicit prices" rather than "exact". - Validation: the plan adds an unknown vendor, a missing price category, an explicit override and token mode.
…unting protocol, unknown-price handling
Review addressed: 619d714All three findings are fixed in the design doc, and each thread has a reply.
Two new open questions bring §11 to seven:
|
… rebuild/merge protocol, per-category pricing; decide Q6-7
|
Second review round addressed in 77fdf49; details are in each thread. Open questions 6 and 7 are now decided in §11: |
| identity (device, inode, and a hash of its first 4 KiB); | ||
| - the read position (byte offset, or last row id), the end of the last complete record only, so | ||
| a half-written line is read again next time; | ||
| - a **revision** number, incremented on every write of the record; |
There was a problem hiding this comment.
[P2] Preserve monotonic revisions across session-cache deletion. The revision lives in the disposable session record, but the project merge accepts only revisions greater than the stored revision. If the project map retains revision 100 and the session record is deleted and rebuilt at revision 1, subsequent updates are rejected until the counter catches up, even when reconstructed usage has increased. Log replacement also says to discard the record without specifying whether its revision is preserved. Please explicitly define a durable per-session revision counter outside the disposable cache, advanced under the session lock and preserved through cache reconstruction and log replacement. The inventory checkpoint written at most once a minute is insufficient on its own because its revision may lag the project map. Add a validation case that deletes only the session cache while retaining a high-revision project entry, appends usage, then verifies that the next hook updates the project total immediately and that a concurrent stale rebuild cannot overwrite it. Spending and pricing wording otherwise address the earlier findings.
There was a problem hiding this comment.
Fixed in 2b2500f (§4.1).
- Durable counter: each session now has a revision counter outside the disposable cache, at
~/.local/state/aegis/budget/sessions/<agent>/<session>.rev. It is the only source of revisions. It is advanced under the session lock and persisted atomically before any write uses the new revision. - What it survives: deleting the session record, the project map or the whole cache, and log replacement or truncation. Step 2 now discards the offset and usage, not the counter. The session record keeps only a copy.
- Lock location: the session lock moved beside the counter, so deleting the cache can never swap out a lock file that is held.
- Counter lost: if the state directory itself is deleted, the counter is recovered under the session lock as one more than the highest revision for that session in any project-map entry or inventory checkpoint, so it never goes backwards against anything a merge compares with.
- Validation: the case you described is added: delete only the session cache while a high-revision project entry remains, append usage, check the next hook updates the total immediately, and check a concurrent stale rebuild cannot overwrite it. It also runs with the log replaced and with the counter deleted.
|
All §11 questions are now decided (695365a), with proposals 1–5 accepted:
§11 is renamed "Decisions", and the |
Design draft for the budget cap (PLAN item 11), targeting v1.0.0. Docs only.
budget.yaml, signed. Its principal needs a newbudgetclass, and an agent can't raise its own limit. It sets per-session and per-project-per-day caps in tokens or estimated USD.Also: PLAN item 11 and
TODO-budget-cap.mdpoint to the design, and PLAN gains a status section dated 2026-10-07.