feat(coding-agent): enforced RLM token budgets with per-depth schedules - #1192
feat(coding-agent): enforced RLM token budgets with per-depth schedules#1192BeeGass wants to merge 21 commits into
Conversation
Adds a per-chat and global RLM token budget with three depth schedules and native enforcement, so recursive subagent trees have a real spend ceiling instead of a prompt-level convention. Schedules distribute a total allowance across depths: - flat: every depth receives the full allowance - geometric: each depth receives factor of the previous depth's allowance - split: a parent reserves factor of its allowance and divides it equally between fanout children, bounding the whole subtree by the root grant Enforcement has two points. Between turns, shouldStopAfterTurn ends the agent loop once a session has generated its allowance; the crossing turn is kept and the loop simply does not start another. Aborting is deliberately avoided since an aborted turn reports stopReason aborted and would not be charged. At spawn time, runRlmChild reserves the child allowance before creating session state and refuses when the schedule cannot fund another child. Budget state flows downward as a value snapshot taken at spawn time, matching how rlmMaxDepth is inherited. A funded child may lower its own allowance but never raise it above the grant it was spawned with. Resolution precedence matches rlmMaxDepth: chat, inherited, global, env, default off. Adds daemon commands get_rlm_token_budget_status and set_rlm_token_budget gated at schema revision 15 with the protocol held at 7.
…isplay Extends /rlm-token-budget in three ways. A spawning model can set a child's allowance with rlm.run(token_budget=N). The request is bounded by what the parent may grant: under split it draws from the same subtree reservation, and under the depth-indexed schedules it may not exceed what the schedule funds at that depth. With no active budget an explicit token_budget starts one for that child's subtree, making budgeting opt-in per delegation. A budget can be a range instead of a single ceiling, as <floor>-<ceiling> or --floor/--ceiling, and rlm.run accepts token_budget=(floor, ceiling). Scheduled allowances are clamped into the range so a decaying schedule cannot starve deep levels. Under split only the ceiling applies to a session's own allowance, since raising to a floor would break the subtree bound; an under-funded child is refused at spawn instead, which preserves the bound and still guarantees no child runs below the floor. A ranged request is funded as far as the reservation allows and refused only when it cannot reach the floor. Budgeted depths now report allowance and usage on subagent snapshots and session summaries, and the agents view renders them as a compact budget used/granted fragment. Reading that metadata degrades when a session cannot report a budget so session listing and attach are never blocked by it.
…sion doubles Session summaries called getRlmTokenBudgetStatus through an optional chain so incomplete test doubles would not throw. That hid the real problem: the doubles were missing a method the real AgentSession always has. The read is direct again, and the doubles in daemon-mode and daemon-session-list project an idle budget the way a real session does.
…ured Covers the default path: no budget state, unchanged spawn behaviour, an agent loop that is never stopped, and a kernel env without budget variables.
…trip Adds the persistence guarantees the max-depth command already has: rollback when chat persistence fails, budget re-resolution when navigating off a chat override, tolerance of a malformed persisted budget, and global settings round-trip including range bounds and clearing.
…s ranges A split floor above the per-child share silently refused every spawn, so `/rlm-token-budget 200k-600k` produced a budget that disabled delegation entirely. That configuration is now rejected when it is set, and the error reports the share the schedule actually provides. Supplying both the positional <floor>-<ceiling> range and --floor/--ceiling silently discarded the flags. Combining the two forms is now an error. Also drops four exports nothing outside the module used and an unreachable clamp on the descendant pool.
…he pool Flooring the per-child share leaves a remainder smaller than one share. The reservation granted it anyway, so the documented default of 1m split 0.5 fanout 3 funded a fourth subagent with 2 tokens: enough to spawn, not enough to finish a turn. A child without an explicit request now requires its whole share, and the remainder is refused. Recomputing the allowance rebuilt the descendant pool from the total, which happens on every /rlm-token-budget call and on branch navigation. A parent could refund live children by re-issuing its own budget. Grants are now deducted permanently. Rebuilding the system prompt on budget changes was dead work, since the prompt carries no budget text, and the branch-reload path assigned the rebuilt base directly and dropped extension prompt modifications. Both rebuilds are removed. Documents that enforcement is per turn, that a parent may fund fewer than fanout children, and that spend accounting does not survive a restart.
The reservation ran before the name-clash, model-resolution, auth-preflight and disposal checks, so an unknown model selector burned a child's whole grant with no way to recover it in-session. Tracking granted tokens made that permanent. The pool is now debited after the last failure path.
…sible The budget stopped a run without telling anyone, forgot its spend, and handed every subagent a fresh copy of the global default. This completes the model. Spend is persisted with the session, so resuming a chat resumes its remaining allowance, and a subagent rehydrated after a daemon restart keeps the grant it was spawned with rather than running unbudgeted. A subagent now resolves only chat and inherited budgets. Falling through to the global default re-seeded each child with a full root-sized budget, so opting a chat out multiplied spend by fanout per depth instead of disabling it. Exhaustion is visible. The agent loop always runs one turn per prompt, so an exhausted session refuses admission instead of burning a charged turn per message, and each refusal re-emits the exhaustion event that the interactive client now renders with the usage and both recovery commands. The active allowance is stated in the system prompt so the model can wrap up rather than being cut off mid-thought. The floor now means what it claims: it is compared against what a funded child may actually spend after reserving its own descendant pool, and a configuration whose share cannot meet it is rejected when set. Ceilings apply to grants, per child shares derive from the pool before any grants so siblings stay equal, and the type guard agrees with the validator so a bad settings file degrades to no budget instead of throwing in every child.
The model was never told it could bound a delegation, so rlm.run's token_budget could not be default behaviour. The recursion prompt now instructs it to size a budget for every child and states that the grant bounds that child and all of its descendants, and the delegation guidance repeats it. No tree-wide configuration is required: with no active budget a grant starts one for that subtree alone, so delegation stays bounded even when the session is not.
…gable A grant was partitioned: a child could spend only 1 - factor of it and the rest was reserved for descendants it might never have. A leaf child funded with 200000 tokens could spend 100000 and strand the other half, which is the wrong default now that agents budget each delegation themselves. A grant is now a single pot. A session may spend all of it, and every token it hands to a child is one it can no longer spend. The subtree bound is unchanged because child grants come out of the same pot, while factor becomes a cap on how much may be delegated rather than a hard partition.
…t model A grant is now what a child may spend, so the separate request and configured floors collapse into one bound and the spendable-vs-grant error branch it fed is unreachable. Restores the untouched runRlmChild step list in the docs and orders the registry validation so no existing line changes.
… coverage Extracts the repeated grant-capturing subagent host and the repeated dim-notice render, merges four pairs of tests that asserted the same behaviour twice, drops a duplicated test, and removes the second copy of the token_budget explanation from the docs. Test contexts are now built on the interactive-mode prototype so shared render helpers resolve as they do in the real class.
6ac2b03 to
4f8e41f
Compare
…ing to A budget capped the main thread as well as its subagents, so setting one stopped the session the user was working in. With a small budget the first turn exceeded it outright, and after a delegation the reported allowance dropped below the number that had just been set, which read as a contradiction. Depth 0 is now never stopped by a budget: the whole budget is the pool it may grant to subagents, and each grant bounds that subagent and everything below it. Only funded subagents are capped, so the exhaustion notice always describes one. Status output now reports granted and remaining tokens rather than a per-session allowance that a delegation had silently reduced.
… a schedule The schedules computed each grant from depth, factor and fanout, which meant the agent doing the delegating could not say what a given child was worth without fighting the formula, and the equal-share rule stranded tokens on siblings that were never spawned. A session now holds a pool and decides for itself how much of it each child is worth. A grant is drawn from the pool and never returned, so the subtree total is bounded by the budget however unevenly the caller allocates it, which is the only guarantee the schedules provided. Floor and ceiling survive as bounds on a single grant. This removes the schedule, factor and fanout knobs along with the per-child share arithmetic and the feasibility checks that existed to keep the schedules honest.
…stale The system prompt stated the remaining pool, but grants happen mid-turn and the prompt was only rebuilt when the budget itself changed. A session that had already granted most of its pool was still told the full figure, so it sized later delegations against a number that no longer existed and discovered the truth only when a spawn was refused. The prompt now states the configured budget and instructs the model to subtract what it has granted, which stays true for the whole turn. Refreshing a live balance per grant was rejected because the system prompt is the cached prefix, so every spawn would invalidate the cache for the rest of the conversation. RLM_TOKEN_SUBTREE_POOL is withdrawn for the same reason: kernel env is fixed at provisioning, so it was absent whenever a budget was set after the kernel started and wrong after the first grant. RLM_TOKEN_ALLOWANCE stays, since a grant is fixed when the session is funded.
…udget code The changelog bullet was appended to the 0.0.1 release entry as well as to Unreleased, editing a frozen section and leaving that release describing a feature it never shipped. That was the only avoidable deletion on the branch. clampToBudgetRange had no production caller and encoded the opposite policy to the one that ships: it raised a below-floor request up to the floor, while the live reservation refuses such a request outright. Its test documented behaviour that never runs, so both are removed. Also drops an orphaned doc comment left by a deleted field, an unused parameter, and comments still describing the schedules, factor, fanout and equal shares that no longer exist.
A concurrent session's scratch test was swept in by a broad git add.
…o guarantee An adversarial review found the headline invariant did not hold. Four defects: The pool was read stale. Grants came from _rlmSubtreePool, which is derived from tokens used, but accounting usage never recomputed it. A subagent granted 100k that had spent 90k still granted a full 100k onward, and chained three deep that spent 270k under a 100k budget. Usage now recomputes the allowance. Branch navigation refilled the pool. Reloading reset granted and used from the branch, refunding grants whose children were still running and un-exhausting a spent session. Those figures describe work that happened, not the branch being viewed, so navigation may now raise them from disk but never lower them. /rlm-token-budget off lifted a parent's grant. Clearing the config dropped the cap entirely and let a subagent grant itself millions, because the reservation path returned early whenever no config was in force. A grant from a parent now stays in force with no local config, and the pool rather than the config decides whether a session may fund a child. Daemon rehydration subtracted delegations twice, because the registry recorded what was left of a grant rather than the grant itself, so a resumed child came back smaller. It now records the grant. The registry reader also dropped completed children whose allowance had reached 0 through full delegation.
Two sentences in the always-on delegation prompt both told the model to size a budget to the task, so every request carried the instruction twice whether or not budgeting was in use. Keeping the one that shows the call signature drops the unconditional cost from 82 words to 46.
|
Update on what's landed since I opened this: 7 commits, net Two of them are design fixes that only showed up once I actually used the thing. Setting a 3000 budget and just talking to the agent printed The per-depth schedules are gone ( Then a second review found the bound wasn't actually holding ( The rest is cleanup: a changelog bullet I'd accidentally appended to the frozen 0.0.1 entry, a dead export encoding the opposite policy to the one that ships, and some duplicated prompt text. Still true: enforcement lands on turn boundaries, so a child funded below one turn's cost overshoots once. And a spawn that dies after its reservation keeps the grant. I tried refunding that and reverted it, since a child that already started may have spent, and at that point the two look the same. |
Recursion depth is already capped by
rlmMaxDepth, but nothing caps what a recursion tree spends. Node count grows asfanout^depth, so a per-agent limit does not bound total cost: three children per node at depth 8 is 9,840 agents, and a fixed per-agent allowance only makes each of them individually modest.This adds a token budget for delegation, enforced by the host rather than requested in a prompt. The budget is what a chat may spend on subagents; the thread the user is talking to is never capped by it.
Usage
Budgeting a single delegation is the default path and needs no configuration:
The grant is drawn from the session's pool and bounds that child and every descendant it spawns. The recursion prompt instructs the model to size a budget for each delegation, so this is doctrine rather than an advanced option.
A tree-wide policy can also be set per chat or globally:
Resolution follows
/rlm-max-depth: chat > inherited > global > env > default off. A subagent stops atinherited, because a child is funded by its parent and falling through to the global default would re-seed every child with a full root-sized budget, multiplying spend byfanoutper depth instead of bounding it.Allocation
There is one rule. A session holds a pool and decides for itself how much of it each child is worth. A grant is drawn from the pool and never returned, so the sum of everything a subtree receives is bounded by the budget however unevenly the caller allocates it:
A range grants as much of its ceiling as the pool affords and refuses the child when even the floor cannot be met, so a child is never funded below the point where it could finish. A child spawned without
token_budget=receives whatever is left, which is why the recursion prompt instructs the model to size every delegation.--floorand--ceilingbound any single grant.Uneven allocation is deliberate: a caller may weight one child far above its siblings, and the pool remains the bound.
Enforcement
Three points, all in the host:
shouldStopAfterTurnends the loop once the allowance is spent. The turn that crossed it is preserved.shouldStopBeforeTurnis gated on!firstTurn), so an exhausted session refuses new prompts. Without this the cap would be a cap plus one fully charged turn for every later message.runRlmChildreserves the child allowance after the last failure path and before any session state exists, then refuses when the schedule cannot fund a child.Aborting is deliberately not used. An aborted turn reports
stopReason: "aborted", which accounting refuses to charge, so aborting on overrun would silently un-charge the turn that tripped the limit.Durability and visibility
Spend is persisted with the session, so resuming a chat resumes its remaining allowance, and a subagent rehydrated after a daemon restart keeps the grant it was spawned with rather than running unbudgeted.
When a budget stops a run the interactive client explains it instead of the session going quiet:
The active allowance is also stated in the system prompt so the model can wrap up rather than being cut off mid-thought, and budgeted rows in the agents view show
budget 12k/500k.Daemon protocol
Adds
get_rlm_token_budget_statusandset_rlm_token_budget, gated atDAEMON_SCHEMA_REVISION16 withDAEMON_PROTOCOL_VERSIONheld at 7, following the additive schema-gated precedent rather than bumping the protocol.DAEMON_SCHEMA_IDis recomputed, the getter is classified read-only, and bothDAEMON_COMMAND_TYPESallowlists are updated. Both compatibility directions are tested: a new client writes nothing to a daemon at the preceding revision and throwsDaemonCapabilityUnavailableErrorbefore any socket write; a daemon at revision 16 accepts.Compatibility
The change is additive:
+2523 / -4across 35 files. The four deletions are therlm.runkwarg contract, the schema revision, its digest, and one test import.With no budget configured the feature is inert, and this is asserted rather than assumed: no budget state, unchanged spawn behaviour, an agent loop that is never stopped, and no new kernel environment variables.
Verification
npm run checkexits 0. The budget-touching suites are 676 tests across 14 files, all passing.The full suite was run on this branch and on unmodified
upstream/mainfor comparison. This branchintroduces no new failing test file. The failures that remain are the same pre-existing ones on both
sides: they spawn real daemons and reach the network.
Two rounds of adversarial review were run against the implementation, the second after the design was
simplified. The second round found that the central invariant did not actually hold, and the fixes for
that are the last commit on the branch. Each has a regression test:
amount. Chained three deep this spent 270k tokens under a 100k budget.
/rlm-token-budget offinside a subagent removed a cap its parent had imposed.back smaller than its parent had funded.
Reviewing this
The commits are ordered to be read in sequence. The first two add the feature; the middle ones fix defects found in an adversarial review of that first draft (a refilling pool, a leaked reservation, a silent stop); the last two are the design corrections that came out of using it, and are the ones worth reading closely:
fix: budget delegation, not the thread the user is talking to, andrefactor: let the caller allocate the budget instead of a schedule, which removes the per-depth schedules in favour of one allocation rule.Worth attention:
_reserveRlmChildAllowanceand_applyRlmTokenBudgetAllowanceinagent-session.ts, which hold the bound; between them they are the whole allocation model.DAEMON_SCHEMA_ID, which is a digest over the wire types and will fail its test if the shapes drift.Known limitations
its grant, so a subagent funded below the cost of a single turn still overshoots once. The budget
bounds how much work is started, not the cost of work already in flight.
a transient runtime failure permanently costs the pool that grant. Refunding it is not obviously safe:
a child that started may already have spent, and the test doubles that abort spawns cannot be
distinguished from real failures at that point. Left as a known cost rather than a risky refund.
/rlm-token-budget, not an automated TUI round trip. The exhaustion notice in particular has beenasserted in tests but not yet observed rendering in a live terminal.
Note
Enforce RLM token budgets with per-depth schedules across subagent sessions
rlm-token-budget.tsas a central module for budget types, validation, token parsing (k/m suffixes), and the/rlm-token-budgetcommand parser.AgentSessionresolves budget config from chat state, inherited config, global settings, or env; refuses to spawn subagents when the pool is drained and emits anrlm_token_budget_exhaustedevent at turn boundaries.get_rlm_token_budget_status,set_rlm_token_budget) gated atminSchemaRevision: 16./rlm-token-budgetslash command for viewing and updating budgets per-chat or globally, and surfaces exhaustion notices in the transcript.Macroscope summarized 074113c.