Repository navigation
perf(runtime): trim per-turn reminders (budget thresholds, minute time, session id once) - #459
Merged
Merged
Conversation
…e, session id once) P1-8: two reminders were spending tokens on every request/turn. --timeout execution budget (request-only, appended per LLM request): - Emit only on the first request, when remaining budget first drops to <=50%, <=25%, <=10% of the total, and on every request inside the final two minutes. Total budget is measured from hook creation (turn prep). - Coarse text: whole minutes above 2 min, 10-second steps inside it. - Drop the "since the previous model request" sentence. - Admission rejection does not consume a threshold. Caller's cancellation timer stays authoritative; hook ordering constraint unchanged. Per-turn <agent-context> system reminder (local runtime): - date: Date#toString() without seconds (minute precision). - The slim (turn >= 2) block omits YOUR SESSION ID when the provider-facing history since the latest compaction already contains a user message with this session's ID. First turn and the first turn after a compaction that dropped it still carry the ID. Full first-turn block and cloud scene are unchanged (flag absent => old behaviour).
hetaoBackend
marked this pull request as ready for review
October 9, 2026 06:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem (P1-8, 0.6.5)
Two reminders spend tokens on every request or turn:
--timeoutbudget reminder (execution-budget-reminder.ts). This is request-only and appended to the end of every LLM request, so its ~40 tokens are never cached:Execution time remaining at request preparation: N seconds. Since the previous model request was prepared: M seconds ....<agent-context>. This block is built inpackages/agent-modules/system-reminder/src/blocks.tsbuildSlimAgentContextBlock. The hosted reminder service inpackages/local-runtime(hosted-agent-capabilities.ts→LocalSystemReminderService→LocalDataCollector) reaches it through v2LocalTurnInputPreparer→reminders.buildSystem. The block is placed at the start of each turn's durable user message. On every turn ≥2 it repeatsYOUR SESSION IDanddate: ${env.date}.env.datewasnew Date(now).toString()(host-memory.ts), which has second precision.Change
(a) Budget reminder: emit only at thresholds
under 10 secondsat the end).canAppendExecutionBudgetReminder) rejects a reminder, that threshold is retried on the next request.(b)
<agent-context>: minute-precision time, session ID only when neededdate:is nowDate#toString()without the seconds, e.g.Fri Oct 09 2026 13:23 GMT+0800 (China Standard Time).compactionSummary(or legacyarchonCompactionmarker). If one exists, it passessessionIdInContext: true→ compat adapter → hostedbuildSystem→LocalReminderSessionInfo→AgentEnv. The slim block then omits theYOUR SESSION IDline.Design notes
Goal. Cut tokens that are re-sent on every request or turn without changing what the model can do, and without touching the stable cached prefix (system prompt, tool list, earlier history).
Why the two reminders are treated differently
<agent-context>is durable. Once written into a turn's user message it never changes, so older turns stay byte-identical and the prefix cache keeps hitting. We only change what is written into new turns. Nothing already in history is rewritten.Budget reminder: why thresholds
Session ID: why "only when not already in context"
Why minute precision for
date:Alternatives considered
Risks
Tests
Run with Node 22.23.3; the repo engines require ≥22.19.
node scripts/run-vitest-suite.mjs capability: 215 files / 5397 tests passed, 18 skipped. This includes:executor.test.ts: rewritten budget tests (threshold steps for 10 min and 2 min budgets, admission retry, skipping straight to the deepest threshold, updated durable-reminder text) and newsessionIdInContextcases (empty history, string and text-part content, compaction summary, legacy marker, kept message after compaction).compat/v1/agent-host.test.ts: checks the flag is forwarded.packages/local-runtime/test/unit/turn-reminder-context.test.ts: minute formatter, slim and full block, cloud unchanged. Registered intest/vitest-suites.jsonandrelease/public-source.json.pnpm typecheckpassed.pnpm lintpassed.node scripts/source-inventory.mjspassed.check:standaloneneedsdist/metafile.jsonfrom a build.Token measurement
Method: a script runs the actual old (origin/main) and new hook code and the real
buildSlimAgentContextBlock, then counts tokens withgpt-tokenizer3.4.0 (already a repo dependency; o200k-family BPE). That is a proxy, not the MiniMax tokenizer, so treat the numbers as relative.--timeout 30m, 40 LLM requests evenly spaced (last one 15 s before the deadline)In (b), the session ID line is about 11 tokens and dropping the seconds saves about 2. Because (b) is durable in history, most of the saving lands on the prompt-cache write and on context growth, not on every request.
Open questions (for Tao)
YOUR SESSION IDbeing present in every turn's reminder, not just somewhere in context?Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.