Three seams #873 measured and did not take: the mid-stream rate test asks the wrong machine, a late first token teaches only a losing arm, and a refusal that cannot name an unknown machine #891

Three things #873 measured and deliberately did not take, because each crosses a file that lane was scoped away from or wants a ruling rather than a patch. They are written down here so they do not live only in a report.

#873 (dev@834bdce26) made the chooser refuse a machine whose expected time to a usable answer is longer than the role's own declared deadline, and made that refusal reach provider.ignore rather than provider.order. All three below are about the half of the same question that lands AFTER the request has gone out, or about the one policy decision that is not ours to make.

1. The mid-stream rate test asks the wrong machine, so a slow machine is never surprising to itself

What happened

2026-09-11 10:02, dev@c9f24f2b6, tag:task node 7. After a bash cd … tool result the next step was served by one machine at roughly 6–8 tokens a second, and ran 219,014 ms — three and a half minutes during which the person watched a cd row and read it as stuck. Two machines on the same model were writing at 90–120 tok/s. From the same call log, that machine's own rate over the previous night:

01:04–01:11   45 rows at 40–105 tok/s (median ~62)
09:33:33      9.8 tok/s
09:37:16      5.9 tok/s   (197 s stream)
09:52:49      6.1 tok/s   (93 s)
09:59:46      7.9 tok/s   (543 s, 4,307 tokens)
10:06:18      6.8 tok/s   (219 s)

The stream was producing tokens throughout, so #853's "a producing stream is never cut" correctly held. What did not happen is a move to one of the machines that were forty times faster.

Why

hazard.keepingUp (internal/lane/control/hazard.go) judges the live per-token gap against h.plan.Gap — the belief about the machine that is answering:

func (h *hazard) keepingUp() bool {
	return !h.plan.Gap.Known() || h.weight < 1 || h.drift/math.Sqrt(h.weight) <= h.z
}

Six tokens a second is not surprising for a machine believed to write at six tokens a second. rateCollapsed is therefore false, the ceiling runs on the ordinary clock, and nothing is done. The test can only ever catch a machine getting worse than ITSELF; it cannot see a machine that is simply far worse than what the request could have had.

(#873 closed the half of this that happens before the send: a machine believed that slow is no longer chosen for a role somebody watches. This is the half that matters once a healthy-looking machine degrades mid-stream, and the series above is five consecutive steps of exactly that.)

The change

control.Plan carries one more field: what the best admissible machine in this request's own frontier is believed to give. lane.PlanFor already holds it — Choice.Frontier carries every candidate's Rate — so nothing new is measured and nothing is fetched.

keepingUp then compares the observed gap's likelihood under the serving machine's belief against its likelihood under that alternative's — a likelihood ratio, not a ratio of medians, so a wide belief is not made to look decisive and a narrow one is not ignored. It stays one predicate, for the reason its own doc comment gives: the two readings of it (which clock the ceiling runs on, and which word the act carries) must always agree.

Replication — deterministic, no model

internal/lane/lanestub: a pool of three machines on one model, one writing at 2 tok/s and two at 90, and a request in a role the table declares Visible. Drive it so the slow machine answers (Profile.Rate: 2, a few hundred tokens). Assert on the controller's Act.

Today: no act before the role's ceiling; the stream runs to completion on the slow machine.

Acceptance — end to end, naming the door

The real door is internal/provider's stream watch driving control.Controller. A lanestub scenario asserting (a) an Act whose Reason is control.RateReason lands while the slow stream is still producing, and (b) the call log's rows for that request name a second, distinct machine. A hazard unit test beside it is fine; instead of it is not.

2. A late first token teaches the ledger only on the LOSING arm of a race

What happened

#853 landed the rule that "a late first token from a named machine teaches the ledger" — the fix for four streams that took 260, 370, 375 and 428 seconds to a first token and taught nothing, so the sheet went on saying that machine answers in eight. The condition was correctly relaxed in streamWatch.sighting (internal/provider/armwatch.go):

if w.served == "" || w.first.IsZero() || w.recovered {
	return lanes.Sighting{}, false
}

But sighting has exactly one caller in the tree — internal/provider/hedge.go, inside a loop that skips the arm that produced the result:

for _, arm := range arms {
	if arm.index == result.index {   // the arm that answered is skipped
		continue
	}
	...
	if sighting, ok := arm.watch.sighting(model, tokens); ok {
		lanes.Default().Ledger().Note(sighting)
	}
}

The skip is justified — "the answer that was used is noted once, by the ordinary path" — and the ordinary path is client.go's epilogue, which sits after every error return in the stream decode loop. A single-arm call is always result.index == 0.

Consequence

A stream with a late first token that was then cut or refused teaches nothing at all: the epilogue is never reached, and settle skips the arm. That is the same shape #853 was written to end, reached by a different door. Two further holes in the same place: knobs.trace.connectionRecovered suppresses the whole measurement after any local connectivity blip, and hedgeRace.abandon (a turn the caller cancelled) writes no sighting at all.

Replication — deterministic, no model

internal/lane/lanestub with Profile.TTFT set past LagTTFT and StallAfter/StallFor arranged so the stream is cut after the first token arrives. One arm, no race. Read lanes.Default().Ledger().Belief(...) before and after.

Today: the belief is unchanged — the machine's published first-token figure survives an answer that proved it wrong.

Acceptance — end to end, naming the door

Through Client.CompleteWithMessages against that stub, not through sighting directly: the belief's first-token median must move toward the observed late one. The likely shape of the fix is that the result arm's sighting is noted by the same door as the losing arms — one place that files every arm's measurement, including the one that errored — rather than by an epilogue that error returns skip.

3. Ruling wanted: may a role's refusal become provider.only rather than order + ignore?

The measurement

2026-09-11 09:33, tag:reflex, mistralai/mistral-nemo — the keypress-lookup tier. The call log's lane field is the machine the preference asked for and served is the machine that answered:

09:33:10   "lane": "DeepInfra"   "served": "DekaLLM"   ms 10746

We asked for one machine and the router used another, because provider.order is a RANKING the router may ignore once allow_fallbacks is true (#850's own sentence). Only provider.ignore takes a machine off the table.

What is closed and what is not

#873 closed the latch: the affinity pin latches onto whichever machine ANSWERED and leads the next order, so that one deviation had become the head of every request for the rest of the session — six of seventeen calls, whole-wait p50 1.8 s → 5.5 s. A refused machine no longer keeps the pin, and once the unknown machine has answered once it acquires a belief, becomes a candidate, and can be refused. Six calls became one.

The first deviation is still possible, and cannot be closed by this mechanism at all. ignoredOf, flooredInto and beyondThePatience all iterate the beliefs this process holds; DekaLLM had no belief for that model and no sheet prior either, so it was never a candidate, so it could never appear in ignore. Meanwhile internal/provider/velocity.go writes AllowFallbacks: &yes on every emitted block. The veto list is built from what we know, while the router picks from what exists.

The question

For a request whose admitted set is non-empty, should the body carry provider.only of the admitted machines instead of order plus ignore?

• The field exists (providerPrefs.Only) and keepTheSetServable already guards the empty-set case, so this is small.
• The cost is real and is why it is a ruling and not a patch. A demand turns a busy pool into a refusal where a preference would have been served by somebody — precisely the trade Never repeat: a body per attempt, a ledger keyed on who served, the sheet's doubts demoted #850 declined when it wrote "a slow answer beats no answer". It would change what happens on every 429-heavy afternoon, not only on the machines this is aimed at.

A middle reading also exists and may be the right one: send only for roles whose declared patience is short and nobody reads the stream (where a slow answer really is no better than none, because the whole of the call is dead time), and keep order + ignore for everything else. That is derivable from the role table's existing columns, so it would need no new configuration — but it is still a change to what a refusal MEANS, and that wants a person to say yes.

Raised out of #873's report. Happy to take any of the three; 1 and 2 are defects with the fix already scoped, 3 needs the ruling first.

🤖 Generated with Claude Code

Drafted with CodeAF · reviewed and owned by the author