Enforce the hard context limit on the full request, with turn rollback - #23
Merged
Conversation
Include tool schemas and tool-call arguments in the live estimate, compact at the soft threshold, refuse when the hard limit still cannot be met, and apply the command prompt cap to run_command results.
Refuse oversized turns before they stick in history/session, force-digest in-flight tool results, and discard incomplete assistant+tool exchanges so the next turn is not permanently stuck over budget.
Force-digesting protected tool results is compaction; with auto_compact off, refuse and roll back the in-flight turn without rewriting earlier tool text.
lavindeep
force-pushed
the
fix/context-budget-estimator
branch
from
July 10, 2026 11:54
44a9201 to
04d65e8
Compare
The hard-limit stop returns an empty assistant Message directly, so the last_reply capture in the tool loop was never read (ruff F841). Remove it and apply the formatter to the hard-limit condition and the recovered check.
Document that the pre-turn and in-loop gates now count tool schemas and refuse (with compaction on or off), the bounded force-digest exception to the protected-tail compaction invariant, and the discard_last_message reconciliation record.
lavindeep
force-pushed
the
fix/context-budget-estimator
branch
from
July 10, 2026 12:23
04d65e8 to
d37d4e8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The context-budget estimate that drives compaction and the hard-limit refusal
only counted the system prompt and history. It ignored the encoded tool
schemas and tool-call arguments — roughly a couple thousand tokens on a
default toolset — so a request that looked under the limit on paper could
still overrun once everything was serialized to the model. There was also no
in-loop guard: a turn that only crossed the limit after several tool results
accumulated was sent anyway, and a refused turn could leave a stuck user
message (or an orphaned assistant tool_call) in history and on
--resume.Change
estimated_prompt_tokens()now includes the liveprofile's encoded tool schemas (
tool_schema_tokens) and per-messagetool-call arguments, closing the undercount.
run_commandresults are heldto the command prompt cap.
both check the full request against
hard_limit_tokens. With automaticcompaction on, the runtime compacts first and refuses only when compaction
cannot recover; with it off, it refuses directly.
message (
discard_last_messagereconciliation record) so--resumedoes notrestore a stuck user turn with no reply. A mid-loop overrun rolls the
in-flight assistant turn back (last assistant message to end of history,
truncate_last_turn) so no orphaned tool_call is re-sent.large enough to overrun on its own inside the tool loop, the runtime
force-digests every tool result — including the normally protected recent
window — and retries once before refusing. This is a bounded, documented
exception to the protected-tail compaction invariant, gated behind
auto_compact; with compaction off the protected tail is never rewritten.Behavior change
Turns that would exceed the hard context limit are now refused deterministically
before the model is called (previously such a request could still be sent).
On refusal the user sees a guidance status message, the offending turn is rolled
back out of history and the session transcript, and the session stays usable
rather than being pinned over budget. This applies whether automatic compaction
is on or off.
Docs
docs/DESIGN.mdupdated in step: the budget section now describes tool-schemacounting and compaction-on/off refusal; §20.2 documents the bounded
force-digest recovery exception; the Session-resume row documents the new
discard_last_messagereconciliation record.Testing
Full local phase gate green at HEAD:
ruff check .— passedruff format --check .— clean (146 files)mypy shellpilot --strict— Success, 83 source filespytest— 1632 passedNew coverage includes: schemas + tool-call args in the estimate; explicit
context setting not capped; refusal when compaction cannot recover; refusal
discarding the recorded user message; mid-loop rollback of the in-flight turn;
mid-loop refusal with
auto_compactoff not digesting the protected tail;discard_last_messageload semantics; andrun_commandhonoring the commandprompt cap.