[waytofarout]

What this changes

Three defaults in internal/config/config.go (and the matching fallbacks in internal/pipeline/stream.go). No logic changes.

Setting	Before	After
SORTITION_POLL_INTERVAL	4s	1s
STREAM_CHUNK_TOKENS	8	32
STREAM_CHUNK_INTERVAL	250ms	100ms

Operators who set these explicitly are unaffected.

1. Poll interval: 4 s to 1 s

Both sortition watchers (SessionWatcher, JobWatcher) tick on this interval. Mainnet blocks are 2 s, so at 4 s a session request or a submitted job waits up to two blocks before the worker looks.

Measured on the mainnet fleet over 7 days at the 4 s default, time from the job's block timestamp to the worker starting stage 2:

Host	p50	p90	Max
g1 (five workers)	0.9 to 2.7 s	2.8 to 3.7 s	5.4 s
A100-Worker-2	2.9 s	5.0 s	5.0 s
A100-Workers	2.1 s	4.1 s	4.1 s
L4-1	2.6 s	2.6 s	2.6 s

The same interval gates session claims: over the same week, session request to claim was 8 s at p50, of which 4 s is the contract's two-block floor and 4 s is the claim transaction.

The nine mainnet workers in our fleet have run with SORTITION_POLL_INTERVAL=1s since 2026-10-13 14:30 UTC with no warnings or errors. Only two jobs had arrived when this was written, both picked up 1.3 s after their block, so the improvement is not yet proven with a real sample. The two testnet workers on A100-Workers have run at 2 s for longer.

Cost: an idle pass is one eth_blockNumber-class call per watcher, so about two cheap RPC calls per second per worker instead of one every two seconds. Each pending session request the worker is not yet eligible for adds a getRequest and an eligibleNow read per pass.

2. Stream batching: by time, not by 8 tokens

The relay keeps a per-consumer token bucket of 1000 messages, refilled at 1000/minute, about 16.7 per second (lightchain-relay internal/ratelimit/limiter.go). Past it, frames are dropped. The chat client ignores the rate-limit notice and only repairs missing text from the terminal complete frame, which is rate-limited too.

A frame every 8 tokens exceeds that refill rate on any fast model. Frame rate and first-frame delay, computed from the coalescer's logic at token speeds measured on the fleet:

Model speed	Frames/s before	Frames/s after	First frame after first token, before	After
10 tok/s	2.5	5.0	300 ms	100 ms
34 tok/s (llama3-70b on g1)	4.2	6.8	206 ms	118 ms
51 tok/s (llama3-8b on an L4)	6.3	7.2	137 ms	118 ms
150 tok/s (llama3-8b on an A100)	18.8	9.3	47 ms	100 ms
232 tok/s (llama3-8b on g1)	29.0	9.2	30 ms	103 ms
320 tok/s	40.0	10.0	22 ms	97 ms

With the new defaults the interval is the binding limit up to about 320 tok/s, where the 32-token cap takes over; the rate reaches the relay's refill rate only above about 530 tok/s.

Trade-off to be aware of: on fast models the first frame goes out about 70 ms later than before (30 to 47 ms becomes about 100 ms). On slow models it goes out sooner. The chat frontend already throttles rendering to 100 ms, so this should not be visible. Sending the very first delta immediately and batching the rest would remove that cost, but it changes the coalescer's behaviour and several handler tests, so it is left out of this PR.

Nothing in settlement changes: chunk frames are throwaway encryptions, and the terminal frame, the blob and completeJob still use the single full-response ciphertext.

I did not measure dropped frames in production; the case for this change is the two rates above, not an observed incident.

Tests

• TestConfig defaults updated for the three values.
• New TestTokenCoalescer_DefaultsStayUnderRelayRefillRate: drives the coalescer with a fake clock at 10, 34, 150, 232 and 320 tok/s and asserts the frame rate stays at or under 10.5 per second. It fails with the old defaults (150 tok/s and up) and passes with the new ones.

Verification

Run locally with Go 1.26.9 in a workspace of this branch plus lightchain-shared-pkg at testnet (58931c3), as CI does:

• go test ./internal/config/... ./internal/pipeline/... ./internal/sortition/...: pass.
• GOOS=linux go build ./... and GOOS=linux go vet ./...: clean.
• gofmt: clean on the four files.
• Not run locally: internal/release and internal/service tests, which do not compile on Windows (unix.Flock). CI covers them.

Not in this PR

• RECEIPT_POLL_INTERVAL is parsed but never read; receipts are polled by bind.WaitMined on a fixed 1 s ticker.
• First-delta immediate flush, as described above.

🤖 Generated with Claude Code

[lightchainaidev]

Review notes, recorded after the merge. None of this blocked it; these are three side effects of the new defaults that the description does not cover.

1. Two retry budgets are counted in poll passes, so they now run out about four times sooner.

• SORTITION_SESSION_RETRY_LIMIT (default 10): JobWatcher gives up on a job whose session is not Active after about 9 s instead of about 36 s. This looks harmless: submitJob requires an Active session and reassignment selects a different worker, so such a session does not come back to the same worker. The shorter wait mostly frees the jobs queued behind it sooner. An operator who wants the old duration can set the limit to 40.
• reseedMaxAttempts (10) in SessionWatcher: the look-back behind the cursor is given up after about 10 s of failing eth_getLogs instead of about 40 s. A worker restarted during a longer log-query failure does not pick up the requests it had discovered before the restart until it is restarted again. A failing head read does not use up attempts, because the pass returns before the look-back.

If either should keep its wall-clock length whatever the poll interval is, counting them in time instead of passes would do it.

2. Slower models send more frames than before.

At 10 tok/s it is 5.0 frames/s (was 2.5), at 34 tok/s 6.8 (was 4.2). The relay bucket is per consumer address, so three parallel streams at 34 tok/s now add up to 20.4 frames/s, above the 16.7/s refill; before it was 12.6/s. The 1000-message burst absorbs about 4.5 minutes of that, so it only matters for a consumer with sustained parallel streams.

3. RPC load per worker is nearer three calls per second than two.

Both watchers read the head every second, and each now runs eth_getLogs on every new block, where at 4 s it ran every other block. Before, the total was about one call per second.

Checked during the review

• Every row of the frame-rate table recomputed from the coalescer logic: the old and new numbers match.
• TestTokenCoalescer_DefaultsStayUnderRelayRefillRate passes at the merged commit and fails at 150, 232 and 320 tok/s with the old defaults put back.
• internal/config, internal/pipeline, internal/sortition and workertest pass against lightchain-shared-pkg at testnet (58931c3).
• Nothing in the README, docs, deploy files or scripts mentions the old values.