You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
call_tool() in the JS sandbox is synchronous and strictly serial: N independent upstream calls
cost the sum of their latencies. A measured run made 31 upstream calls in 20.6s — nearly all of
it waiting on I/O that has no ordering dependency. Fan-out patterns (fetch 20 PRs, read 15 issues,
poll 8 servers) are exactly what the code-execution feature exists for, and they are the ones that
pay the most.
There is no way to work around this in userland: the sandbox deliberately ships no timers
(setTimeout/setInterval removed) and an ES5.1-only stdlib, so there is no Promise or event-loop
story to build concurrency on. The primitive has to come from the host.
Proposed solution
Add a host-provided batch primitive rather than JS async:
varresults=call_tools([{server: "github",tool: "get_pull_request",args: {owner: "o",repo: "r",pullNumber: 1}},{server: "github",tool: "get_pull_request",args: {owner: "o",repo: "r",pullNumber: 2}}]);// results[i] is the same {ok, result} | {ok, error} envelope call_tool returns,// in input order, one entry per request, never short-circuited.
Details worth pinning down in review:
Error semantics: one failing call must not fail the batch — each slot carries its own {ok:false, error}. This is the main reason to prefer an explicit batch API over implicit
parallelism.
Concurrency bound: a max_parallel option (config default, per-call override), and the batch
must respect whatever per-server request queueing exists — see closed [Feature]: request queueing / concurrency limit for multi-user deployments #955 on request queueing /
concurrency limits, which is the counterweight this design has to honour rather than bypass.
Accounting: each element counts against max_tool_calls individually.
Timeout interaction: the batch is bounded by the overall execution timeout; note that the CLI
daemon path currently caps at ~2s regardless of --timeout (filed separately), which would mask
most of the benefit until fixed.
This keeps the sandbox ES5.1 and timer-free while turning sum(latency) into max(latency) for the
common fan-out case.
Which edition?
Both.
Alternatives considered
Promises / async-await in the sandbox. Requires an event loop and timers in goja — much larger
change, and the timers were removed deliberately.
Callers issuing N separate code_execution calls in parallel from the client. Loses the entire
point of the sandbox (a single round trip with local aggregation) and multiplies script-transfer
cost by N.
Implicit auto-parallelisation of consecutive call_tool() statements. Not statically safe;
ordering and side-effect dependencies are undecidable at that level.
I searched existing issues and discussions and did not find a duplicate
Problem / motivation
call_tool()in the JS sandbox is synchronous and strictly serial: N independent upstream callscost the sum of their latencies. A measured run made 31 upstream calls in 20.6s — nearly all of
it waiting on I/O that has no ordering dependency. Fan-out patterns (fetch 20 PRs, read 15 issues,
poll 8 servers) are exactly what the code-execution feature exists for, and they are the ones that
pay the most.
There is no way to work around this in userland: the sandbox deliberately ships no timers
(
setTimeout/setIntervalremoved) and an ES5.1-only stdlib, so there is no Promise or event-loopstory to build concurrency on. The primitive has to come from the host.
Proposed solution
Add a host-provided batch primitive rather than JS async:
Details worth pinning down in review:
{ok:false, error}. This is the main reason to prefer an explicit batch API over implicitparallelism.
max_paralleloption (config default, per-call override), and the batchmust respect whatever per-server request queueing exists — see closed [Feature]: request queueing / concurrency limit for multi-user deployments #955 on request queueing /
concurrency limits, which is the counterweight this design has to honour rather than bypass.
max_tool_callsindividually.daemon path currently caps at ~2s regardless of
--timeout(filed separately), which would maskmost of the benefit until fixed.
This keeps the sandbox ES5.1 and timer-free while turning
sum(latency)intomax(latency)for thecommon fan-out case.
Which edition?
Both.
Alternatives considered
Promises / async-await in the sandbox. Requires an event loop and timers in goja — much larger
change, and the timers were removed deliberately.
Callers issuing N separate
code_executioncalls in parallel from the client. Loses the entirepoint of the sandbox (a single round trip with local aggregation) and multiplies script-transfer
cost by N.
Implicit auto-parallelisation of consecutive
call_tool()statements. Not statically safe;ordering and side-effect dependencies are undecidable at that level.
I searched existing issues and discussions and did not find a duplicate