Repository navigation
perf(agent): end the agent loop once every required tool has been called - #72
Conversation
An agent whose output is delivered through a required tool has nothing left to say once that tool has run, but the loop only returns when the model answers with no tool calls. So every such agent spends one further request that exists only to confirm it is finished. That request is the most expensive one the agent makes: the transcript is never longer than it is at the end, and the whole of it is re-sent to be told there is nothing more to do. A review runs a plan agent, N sub agents and a writing agent, so the pipeline pays for it N+2 times, in latency and in input token alike. Adds endWhenRequiredToolsCalled, off by default so nothing changes for an agent that is meant to keep working after calling a required tool, and turns it on for the review sub agents and the writing agent, where the required tool is the agent's output by definition. The reply and issue agents deliver their output the same way and would benefit identically, but they are left alone here so the change can be judged on the review pipeline first. Refs seoes#70
|
Thanks for this — a tight, well-scoped change that removes a wasted confirmation turn from the two review agents without touching the other required-tool call sites. Overview This PR adds an opt-in Good points
Additional Notes
|
|
Great. I was aware of the fact that the agent loop always produces the final answer but didn't make any update for it It'll make much faster agent loop thank you for made this change! |
Refs #70.
The problem
runAgentLoopreturns only when the model answers with no tool calls. But several agentsdeliver their entire output through a tool:
submit_review_handoffpost_pull_request_commentandevaluate_pull_requesttheir own required tool
Once that tool has run, there is nothing left for the model to say. The loop nonetheless
sends one more request, purely to receive a message with no tool calls so it can return.
That request is the most expensive one the agent makes. The transcript is never longer than
it is at the end of a run, and all of it is re-sent to be told there is nothing more to do.
A review is a plan agent, N sub agents and a writing agent, so the pipeline pays for it
N+2 times, in latency and in input token alike.
The change
A new loop option,
endWhenRequiredToolsCalled. When it is set, the loop returns as soonas every tool in
requiredToolListhas been called at least once, immediately after thatstep's tool results are appended.
It is off by default, so nothing changes for an agent that is meant to keep working
after calling a required tool. It is turned on for the review sub agents and the writing
agent, where the required tool is the agent's output by definition.
The reply and issue agents would benefit identically and the one line is the same, but I
have left them alone so this can be judged on the review pipeline first.
finalMessagebecomes the assistant content from the step that called the tool, which maybe null. Nothing in the codebase reads
finalMessageoutside the loop's own tests, andboth review services discard it, so this is not a behavioural change for any caller.
Why not just stop requiring the extra turn for everyone
Because the loop cannot tell, in general, whether a required tool is an agent's output or
one step of its work. The existing mechanism only enforces that the tool is called before
finishing. An opt-in keeps that distinction with the caller, who knows which it is.
Tests
Five new cases in
loop.test.ts, all counting requests rather than asserting on messages,since the saved request is the point:
Three of the five fail without the loop change; the other two assert unchanged behaviour
and pass either way, which is intended.
Full suite is 71 pass, 0 fail.
tsc --noEmitonapps/apiis clean. Prettier is clean onevery file this touches except
writing.service.ts, which already fails onmainfor anunrelated import wrap; I have left that alone rather than add unrelated churn to this
diff.
What this does not fix
The two larger causes in #70 are untouched:
maxStepsdefaulting to 100 per agent withno production caller passing it, and sub agent fan-out width, which on my instance tracks
duration more closely than diff size does. This change removes a fixed N+2 requests from
every review; it does not bound the variable part.