Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ results/*
!results/benchmark.json
!results/latency.md
!results/latency.json
!results/budget-latency.json
!results/llm-external.md
!results/llm-cache-naive.jsonl
!results/llm-cache-aware.jsonl
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,16 @@ test on each agent are in.
can block them (Claude Code and Codex `UserPromptSubmit`, Gemini CLI `BeforeAgent`); a warning
is shown once per session per threshold (`systemMessage`; an OpenCode toast; stderr for Copilot
CLI). A `budget.yaml` that cannot be used (bad signature, shape, authority) fails closed.
- `aegis budget status|check|reset` (design §7): today's project total and its sessions, rebuilt
from the logs (`--json`); `check` exits 0 under, 3 over, 65 for an unusable `budget.yaml`, 66
for none; `reset --day` acknowledges an incomplete day's history as it stands, with a marker
signed by the policy key that records the acknowledged state, so `on_unknown_log: deny` stops
denying for exactly that (later problems count again; an agent without the key cannot reset).
- `docs/budget.md`; `aegis init` writes a signed `budget.example.yaml`.
- `scripts/budget_latency.py`: the budget adds ~9 ms to a hook call (74 vs 65 ms median,
`results/budget-latency.json`), under the 20 ms target. The price table ships as
`prices.json` generated from `prices.yaml` (`scripts/sync_prices.py`; a test keeps them equal)
and is read only from the package: parsing the YAML on every call had cost ~15 ms.
- `aegis install` adds the budget hooks when the policy directory that applies holds a
`budget.yaml` (`--budget` / `--no-budget` to choose): the tool hook sees every tool call and
the prompt hook is added where the agent has one. The Claude Code plugin and the Gemini
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,7 @@ of anything you care about.
| [`docs/constraints.md`](docs/constraints.md) | Writing constraints, metadata vocabulary, authority policy, environment mapping |
| [`docs/configuration.md`](docs/configuration.md) | Configuration/config-dir discovery, signing, source verification, the identity model (`agents.yaml`), rate limits & ledger |
| [`docs/server-side.md`](docs/server-side.md) | Preview: compiling the policy to AWS Service Control Policies scoped to agent identities |
| [`docs/budget.md`](docs/budget.md) | The session budget cap: per-session and per-project-per-day limits in estimated USD or tokens, across Claude Code, Codex, Gemini CLI, OpenCode and Copilot CLI |
| [`docs/benchmark.md`](docs/benchmark.md) | Benchmark methodology, full results table, real LLM baselines, agent-harness baselines, corpus, adversarial suite |
| [`docs/CONTRIBUTING.md`](docs/CONTRIBUTING.md) | Adding a new parser or tool |
| [`docs/dev/PLAN.md`](docs/dev/PLAN.md) | Design plan and open gaps |
Expand Down
21 changes: 21 additions & 0 deletions data/budget.example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# The session budget cap (docs/budget.md). Copy to budget.yaml, set your own limits, and sign
# it with `aegis sign`; its principal must hold the 'budget' class in authority.yaml. Until
# it is named budget.yaml it does nothing. Then reinstall your agents' hooks (`aegis install
# <agent>`) so they see every tool call, and check with: aegis budget status
version: 1
principal: admin # must hold the 'budget' class in authority.yaml
unit: usd # usd (estimated from list prices) | tokens; required
session:
limit: 20 # per agent session
warn_at: 0.8 # warn once at 80%
project_day:
limit: 100 # every measured session in this project, per day
warn_at: 0.8
tz: UTC # the day boundary
agents: [claude, codex, gemini, opencode, copilot] # only listed agents are measured
on_unknown_log: allow # allow (warn) | deny: usage Aegis cannot verify
on_unknown_price: estimate # estimate (warn) | deny: models with no price
pricing: {} # signed overrides, USD per million tokens:
# claude-opus-5-5: {input: 4, output: 20, cache_read: 0.2, cache_write: 8}
copilot:
premium_requests: 50 # Copilot CLI reports premium requests, not tokens
2 changes: 2 additions & 0 deletions data/budget.example.yaml.sig
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
aegis-sig-v1
d8ad9e56f8548c60028c05c17ce2f46ca190127cbafde00669faef484f3eaea4
161 changes: 161 additions & 0 deletions docs/budget.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Session budget cap

A coding agent stuck in a loop, or fanning out sub-agents, spends money (API keys) or quota
(subscriptions) with nothing to stop it. Aegis can cap that spend per agent session and per
project per day, in estimated dollars or in tokens, across Claude Code, Codex, Gemini CLI,
OpenCode and Copilot CLI (premium requests). The cap is written into a **signed** policy
file, so an agent cannot raise its own limit by editing a file.

← back to the [README](https://github.com/moneytool/aegis-devops#readme) · design: [`dev/DESIGN-v1.0-budget-cap.md`](https://github.com/moneytool/aegis-devops/blob/main/docs/dev/DESIGN-v1.0-budget-cap.md)

## What it guarantees, and what it does not

> Aegis denies covered tool calls and prompts when the usage available at that hook reaches
> the limit. It does not impose a finite bound on additional model spending: in-flight
> requests, delayed usage records, retries after a denial, and tool-free activity can add
> cost.

Hooks run before tool calls and prompts, never before a model request, and an agent writes a
request's usage to its log only after the request has been paid for. So a session can end
**above** its limit — by the request that crossed it, by the model's replies to the denials,
by sub-agents that were already running, and, in OpenCode and Copilot CLI (which have no prompt
hook that can block), by chat that uses no tools. `aegis budget status` shows any usage over
a limit as **overshoot**. If you must never exceed an amount, set the limit below it, and put a
spend limit in front of the model requests themselves as well (a gateway, your vendor's
spending limits); those are separate layers whose guarantees you need to check.

The budget is a guard against loops and mistakes. An agent runs as you and can rewrite its own
session log; Aegis does not defend against an agent that does that on purpose.

## Set it up

```bash
aegis init .aegis # once; writes budget.example.yaml among the examples
cp .aegis/budget.example.yaml .aegis/budget.yaml
$EDITOR .aegis/budget.yaml # your limits
aegis sign --key file:KEY .aegis/budget.yaml
aegis install claude # and codex, gemini, opencode, copilot: reinstall
aegis budget status
```

Reinstalling matters: with a `budget.yaml` in the policy directory, `aegis install` widens each
agent's hook from shell commands to **every tool call** and adds the prompt hook where the
agent has one (`--budget` / `--no-budget` choose explicitly). The Claude Code plugin and the
Gemini extension already carry those entries; they start no extra process in projects without
a budget.

## `budget.yaml`

```yaml
version: 1
principal: admin # must hold the 'budget' class in authority.yaml
unit: usd # usd | tokens — required
session: {limit: 20, warn_at: 0.8} # per agent session
project_day: {limit: 100, warn_at: 0.8, tz: America/Chicago} # all sessions in the project, per day
agents: [claude, codex, gemini, opencode, copilot] # only listed agents are measured
on_unknown_log: allow # allow (warn) | deny
on_unknown_price: estimate # estimate (warn) | deny — usd only
pricing: # signed overrides, USD per million tokens
claude-opus-5-5: {input: 4, output: 20, cache_read: 0.2, cache_write: 8}
copilot: {premium_requests: 50, warn_at: 0.8} # per Copilot CLI session
```

- **Who may change it.** The file is signed like every policy file, and its `principal` must
hold the `budget` class (the example `authority.yaml` gives it to `admin`). A file that was
edited after signing, or whose principal lacks the class, is refused — and inside a project
with a budget the hook then **denies** every covered call with a message saying so, rather
than run without the budget you asked for.
- **Errors, never guesses.** Unknown fields, a missing `unit`, a limit that is not a finite
positive number, `warn_at` outside (0, 1), an unknown time zone or agent, a price override
missing a category: all load errors.
- **`tokens`** counts input, output, cache-read and cache-write tokens together, and ignores
prices.
- **Copilot CLI** writes token counts only at shutdown, so it is measured in premium requests
and only when `copilot.premium_requests` is set.
- **Cursor and VS Code** expose no token data and are not measured.

## What happens near and at the limit

| | Tool calls | New prompts | You see |
|---|---|---|---|
| Below `warn_at` | allowed | allowed | nothing |
| Past `warn_at` | allowed | allowed | one warning per session per threshold |
| At the limit | **denied**, every tool | **denied** where the agent can block them | what was spent, which limit, how to continue |

The infrastructure policy still decides first: a command the policy blocks stays blocked
whatever the budget says. To continue past a limit: start a new session (session limit), wait
for the next day in `tz` (project limit), or raise the limit in `budget.yaml` and re-sign it.

| Agent | Tool hook | Prompt hook | Warning shows as |
|---|---|---|---|
| Claude Code | `PreToolUse`, every tool | `UserPromptSubmit` | a message in the transcript |
| Codex CLI | `PreToolUse`, every tool | `UserPromptSubmit` | a UI warning |
| Gemini CLI | `BeforeTool`, every tool | `BeforeAgent` | a message in the terminal |
| OpenCode | plugin, every tool | — | a toast |
| Copilot CLI | `preToolUse`, every tool (premium requests) | — (it cannot block) | stderr only |

## Where the numbers come from

No hook payload carries token usage, so Aegis reads each agent's own session log, incrementally:
Claude Code's transcript (and its sub-agents'), Codex rollouts, Gemini CLI chats, OpenCode's
database (read-only; only its session and message tables), and Copilot CLI's events. A
sub-agent is counted inside its parent session. Nothing is sent anywhere.

**Dollars are estimates** from a dated table of list prices (`aegis_core/budget/prices.yaml`;
Anthropic, OpenAI and Google). On a subscription they are an API-equivalent figure, not your
bill; `unit: tokens` may suit you better there. A model the table does not know is priced, per
category, at the highest rate of its vendor (or of the whole table) — a guess that can still be
low — and flagged; with `on_unknown_price: deny` its sessions are denied until you add a signed
`pricing:` entry for it.

**The day** is the calendar day in `project_day.tz`, from the time each log record carries.

**What Aegis keeps** is derived from the logs and can be rebuilt: a cache under
`~/.cache/aegis/budget/`, and a small durable state under `~/.local/state/aegis/budget/` (a
per-session revision counter, locks, and a list of the sessions seen each day, so a rebuild can
tell when history is missing). Deleting the cache resets nothing while the logs exist.

### When usage cannot be verified

Some usage can only be counted as a lower bound: a log Aegis cannot find or read, records it
does not recognise, usage with no recorded time, a project day rebuilt after sessions' logs
were deleted, or the first day of a budget when other sessions had already run that day.
`on_unknown_log` decides what that means:

- `allow` (default): a warning, once; the lower bound is used.
- `deny`: covered calls are denied. A session's own unreadable log stays denied (start a new
session). For an incomplete **project day**, `aegis budget reset --day` acknowledges that
day's history **as it stands**: it records the day's verification state at that moment (how
complete the history is, and for each session that is a lower bound: whether its log is
missing, how many records could not be read, how much usage has no recorded time) and is
signed with the
policy key, so an agent cannot write one. Exactly that is excused; anything that becomes
unverifiable afterwards — a new session with an unreadable log, another unreadable record in a
session already acknowledged, more sessions without a log — counts again. Otherwise it clears at the next day.

## Commands

```bash
aegis budget status [--project DIR] [--agent A] [--json]
aegis budget check [--project DIR] [--agent A --session ID]
aegis budget reset --day [YYYY-MM-DD] [--project DIR] [--key KEY]
```

- `status` rebuilds today's project total from the logs, then shows it against the limit, each
session that contributed (today's usage, its session total against the session limit, and
whether it is a lower bound), estimated prices and the policy's warnings.
- `check` exits **0** under budget (a warning included), **3** over, **65** if `budget.yaml`
cannot be used, **66** if there is none: for CI and scripts, including headless agent runs.
It rebuilds the project day from the logs too; with `--agent` and `--session` it also checks
that session's own limit.
- `reset --day` rebuilds the day, then writes the signed acknowledgement of its current state
described above (default: today in the policy's `tz`); it needs the signing key and refuses
`--insecure`.

## Cost

Measured with [`scripts/budget_latency.py`](https://github.com/moneytool/aegis-devops/blob/main/scripts/budget_latency.py)
([`results/budget-latency.json`](https://github.com/moneytool/aegis-devops/blob/main/results/budget-latency.json)):
the budget adds about **9 ms** to one hook call (Apple M4 Pro, Python 3.12: 74 ms median with a
budget, 65 ms without), against a design target of 20 ms. Projects without a `budget.yaml` pay
nothing.
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ ticket or chat message (prompt injection, context poisoning) cannot become polic
- [Writing constraints](constraints.md) — rules, scopes, authority, environments
- [Configuration](configuration.md) — config directory, signing, Git sources, `agents.yaml`
- [Server-side enforcement](server-side.md) — AWS SCPs and Kubernetes admission policies
- [Session budget cap](budget.md) — per-session and per-day spend limits across agents
- [Benchmark](benchmark.md) — Aegis against OPA and LLM self-checks on poisoned rules

## Project
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ version = {attr = "aegis_core.__version__"}
where = ["src"]

[tool.setuptools.package-data]
aegis_core = ["_examples/**/*", "integrations/*", "compile/actions/*", "budget/*.yaml"]
aegis_core = ["_examples/**/*", "integrations/*", "compile/actions/*", "budget/*.yaml", "budget/*.json"]

[tool.ruff]
line-length = 100
Expand Down
16 changes: 16 additions & 0 deletions results/budget-latency.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"measured": "2026-10-10",
"runs": 40,
"python": "3.12.12",
"platform": "Darwin arm64",
"hook_without_budget": {
"median_ms": 64.7,
"p95_ms": 67.3
},
"hook_with_budget": {
"median_ms": 74.2,
"p95_ms": 76.7
},
"added_median_ms": 9.5,
"target_ms": 20
}
113 changes: 113 additions & 0 deletions scripts/budget_latency.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
"""What the budget cap adds to one ``aegis hook`` call (design v1.0 §4: target
under 20 ms).

Times ``python -m aegis_core.cli hook claude`` as the agent runs it (a new
process per tool call) on a non-shell tool call, in two otherwise identical
projects made with ``aegis init``: one with a signed ``budget.yaml`` and a
Claude Code transcript to read, one without. Runs are interleaved so both see
the same machine load, after a warm-up run, and with bytecode caching on (a
pip install ships compiled bytecode; ``PYTHONDONTWRITEBYTECODE`` would make
every run recompile aegis and inflate both numbers).

python scripts/budget_latency.py [--runs 40] [--out results/budget-latency.json]
"""

from __future__ import annotations

import argparse
import json
import os
import platform
import statistics
import subprocess
import sys
import tempfile
import time
from datetime import UTC, datetime
from pathlib import Path

ROOT = Path(__file__).resolve().parent.parent


def _setup(base: Path) -> tuple[dict[str, str], dict[str, str]]:
sys.path.insert(0, str(ROOT / "src"))
from aegis_core.config import init_config_dir
from aegis_core.signing import load_key, sign_file

home = base / "home"
projects = {"budget": base / "with-budget", "none": base / "without-budget"}
payloads = {}
now = datetime.now(UTC).isoformat().replace("+00:00", "Z")
for name, project in projects.items():
init_config_dir(project / ".aegis")
if name == "budget":
budget = project / ".aegis" / "budget.yaml"
budget.write_text("principal: admin\nunit: usd\nsession: {limit: 20}\n"
"project_day: {limit: 100}\nagents: [claude]\n")
sign_file(budget, load_key(f"file:{project / '.aegis' / 'example-signing.key'}"))
log = home / ".claude" / "projects" / f"-{name}" / "s.jsonl"
log.parent.mkdir(parents=True, exist_ok=True)
recs = [{"type": "user", "sessionId": "s", "cwd": str(project), "timestamp": now}]
recs += [{"type": "assistant", "sessionId": "s", "requestId": f"r{i}", "timestamp": now,
"message": {"id": f"m{i}", "model": "claude-opus-5-5",
"usage": {"input_tokens": 1000, "output_tokens": 100}}}
for i in range(200)]
log.write_text("".join(json.dumps(r) + "\n" for r in recs))
payloads[name] = json.dumps({"session_id": "s", "transcript_path": str(log),
"cwd": str(project), "tool_name": "Edit",
"tool_input": {"file_path": "x"}})
env = {k: v for k, v in os.environ.items() if k not in (
"PYTHONDONTWRITEBYTECODE", "AEGIS_CONFIG_DIR", "XDG_CACHE_HOME", "XDG_STATE_HOME")}
env.update({"HOME": str(home), "PYTHONPATH": str(ROOT / "src")})
return payloads, {**env, "_budget": str(projects["budget"]),
"_none": str(projects["none"])}


def main() -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--runs", type=int, default=40)
ap.add_argument("--out", default=str(ROOT / "results" / "budget-latency.json"))
args = ap.parse_args()
with tempfile.TemporaryDirectory() as tmp:
payloads, env = _setup(Path(tmp))
cwd = {"budget": env.pop("_budget"), "none": env.pop("_none")}
cmd = [sys.executable, "-m", "aegis_core.cli", "hook", "claude"]

def run(name: str) -> float:
start = time.perf_counter()
proc = subprocess.run(cmd, input=payloads[name], text=True, capture_output=True,
env=env, cwd=cwd[name], check=False)
if proc.returncode != 0:
raise SystemExit(f"hook failed ({name}): {proc.stderr}")
return (time.perf_counter() - start) * 1000

for name in payloads: # warm-up: bytecode, price-table cache, session record
run(name)
samples: dict[str, list[float]] = {"budget": [], "none": []}
for _ in range(args.runs):
for name in samples:
samples[name].append(run(name))

def summary(xs: list[float]) -> dict[str, float]:
ordered = sorted(xs)
return {"median_ms": round(statistics.median(xs), 1),
"p95_ms": round(ordered[max(0, int(len(xs) * 0.95) - 1)], 1)}

with_b, without = summary(samples["budget"]), summary(samples["none"])
result = {
"measured": datetime.now(UTC).date().isoformat(),
"runs": args.runs,
"python": platform.python_version(),
"platform": f"{platform.system()} {platform.machine()}",
"hook_without_budget": without,
"hook_with_budget": with_b,
"added_median_ms": round(with_b["median_ms"] - without["median_ms"], 1),
"target_ms": 20,
}
Path(args.out).write_text(json.dumps(result, indent=2) + "\n")
print(json.dumps(result, indent=2))
return 0


if __name__ == "__main__":
sys.exit(main())
Loading
Loading