Batch: say what it costs and how long it takes - #19
Merged
Conversation
llms.txt promised that "50 calls in one request cost ONE rate-limit
token instead of fifty", which is half true and reads as if a batch were
free. It saves the FREQUENCY limit. It does not save the work.
Measured on a live portal. The resource-intensity counter (operating) is
kept per method, and a batch is charged on its own: after a batch of 20
crm.deal.add, crm.deal.add's own counter had risen by exactly the ONE
add that went through a separate call, while the batch counter gained
about 4 seconds for each of two consecutive 15-command batches. Watching
the write method's counter therefore shows nothing while the cost is
real.
Nowhere did the docs say how long a batch runs. The commands execute one
after another inside a single HTTP request: 50 crm.deal.add took 28.5
seconds, 20 took 12.0, 15 took 8.1 — about 0.6s each. That is longer
than most default timeouts, and the failure is the expensive kind. An
http.Client{Timeout: 30s} cuts the connection after the portal has
already created part of the deals: an ambiguous failure the SDK rightly
does not replay, with the ids of everything created lost alongside it.
The behaviour the text now leans on is pinned by a test rather than left
to the prose: a batch cut short by a client timeout is sent once, comes
back with an error and no result.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
A live run (50 deals in one batch) caught the documentation in a half-truth.
1. "ONE rate-limit token instead of fifty" — true of the frequency limit only
llms.txtsaid:The sentence reads as "a batch is free". It saves nothing on the resource-intensity limit: the portal still runs all fifty commands and still charges their time.
Measured on a live portal. The
operatingcounter is kept per method, and a batch is charged on its own:That is, the write method's counter looks almost untouched after a batch. Whereas the counter of
batchitself, over two consecutive batches of 15 creates:The cost is there, it is just in a different bucket. The wording now is: a batch moves the pressure from one limit to the other; it does not remove it.
2. Nowhere was it said how long a batch takes
Measured: 50
crm.deal.add= 28.5 seconds in one HTTP request (20 commands — 12.0 s, 15 commands — 8.1 s), roughly 0.6 s per command, the commands running sequentially — the envelope'sdate_startanddate_finishare a whole duration apart.The consequence is sharp and it costs data:
The connection is cut after the portal has created part of the deals. That is an ambiguous failure — the SDK does not replay it, and rightly so: a replay would create them a second time — but the result is lost, and with it the identifiers of everything already created. The SDK's HTTP client has no timeout by default, so this catches only the code that set one.
Where it was fixed
llms.txt(item 6 + a new trap under "Traps that cost data"),README.md(the batch section), the godoc ofBatchandCallBatch,doc.go,CHANGELOG.md.CHANGELOG: added to the
0.1.0section, into therest.batchitem, as a description of what is true — not as a "Fixed". 0.2.0 is not published yet, the wrong wording never went out; the command-order bug was handled exactly the same way last time.Checks
go build ./...go vet ./...gofmt -l .— no outputgo test -race ./...Compatibility
Tests
The PR is a documentation one, but one claim in the text is worth being held by a test rather than by prose:
TestBatchCutShortByAClientTimeoutLosesWhatItCreated. The server sleeps longer than the client timeout and only then answers with a result carrying an identifier — that is, the entities have already been created. What is checked is that the caller gets an error and anilresult (the identifiers are unreachable) and that the batch was sent once: a replay would create everything a second time. Run with-count=20 -race— it does not flake.The existing
TestBatchIsNotRetriedOnAmbiguousFailurecovered a 502; this one covers precisely a connection cut on a long batch, the very shape the documentation now warns about.