perf(sparse_moe): widen the Q5 routed-down Rows2 window to its measured crossover

[MichaelDementii]

Re-measured on a16b6442. The diff changed, and so did the answer

The review bot's P1 on this PR was right on substance: an eleven-token threshold knowingly leaves
the worse schedule in the product at widths where the sweep in this very report shows Rows2
winning. I agreed, re-ran the crossover on the current head, and the number moved. This
revision therefore replaces both the diff and the numeric spine of the text.

• New base: a16b6442 (docs: organize performance reports and update 35b results), the
  head of origin/master at the time of the re-measurement. It is 487f8977 plus exactly one
  commit, and that commit touches 13 files, all of them README.md, docs/, model-cards/,
  eval/README.md and tools/bench/README.md. tests/CMakeLists.txt is byte-identical between
  the two, so the registered test set does not move.
• New diff. Not "raise the Q5 threshold to eleven" any more. The codec branch is removed and a
  single measured threshold of 19 replaces both arms.
• Two measurement sessions, and every number below says which one it belongs to. The 2026-09-03
  sweep and the whole verification layer of the previous revision were taken on a140e7ae /
  ad0f3d38, against a diff that set eleven. The 2026-09-07 sweep, the bitwise gate, the
  end-to-end ladder, the round step and the targeted ctest were taken on a16b6442, against the
  diff proposed here, i.e. at the constant 19. Nothing from the old session is relabelled as if it
  had been taken at 19; where the two disagree, both are printed with the base each belongs to.
• Not everything from the old session was re-taken. The graph-step ladder and the
  cuobjdump resource census were not repeated on a16b6442. They are marked where they appear,
  and again under Limitations.

Level of the claim: schedule (kernel selection), confirmed end to end. On this artifact the change is
inert at T >= 20 and live at T <= 19, through two entries: a decode round of 6..19 columns on
the Q5 arm and 12..19 on the Q6 arm, and a prefill chunk of 2..19 tokens on either. The second
entry is why T <= 5 is not inert either.

Environment

RTX 5090, sm_120a. Release, -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120a -DNINFER_BUILD_BENCHMARKS=ON, CUDA 13.1.115 with host compiler GNU 13.3.0 (logs/cmake.log:1-3).
Artifact Qwen3.6-35B-A3B; the engine's own summary reports the resident weight set as 19.6 GiB
with no speculative head, 20.0 GiB under --spec dflash and 20.4 GiB under --spec mtp
(weights_capacity_bytes = 21 038 469 632 / 21 448 523 264 / 21 936 403 968 in the ladder CSVs).
Base origin/master a16b6442.

Every measurement runs under a GPU-exclusive lock that refuses to start when a foreign compute
process holds the card. In this session the card witness (nvidia-smi sampling of SM clock, memory
clock, temperature and power) is started after the lock is taken and killed before it is
released, so the witness window can no longer include time the card was not ours. Over the sweep the
SM clock reads median 2872 MHz, mode 2902 MHz, maximum 2910 MHz, at up to 66 C and up to
538.03 W - 4031 samples across the three witness logs, 23 of them above 535 W.

The operator instrument is ninfer_sparse_moe_bench (bench/ops/sparse_moe_bench.cu) at
--cache cold --execution graph --distribution trace-like with the default seed. Every pass is its
own process under its own lock acquisition; pass 0 of each arm is discarded and the median of the
rest is taken. 520 raw files (raw/ in the attached archive: 24 each for blocks F, G, H, K, M, N
and Z, 272 single points, 80 single-point null-control runs).

Corrected before submission. An earlier draft of this section carried three numbers that the
session does not support: "Artifact Qwen3.6-35B-A3B, 21.2 GiB", "up to 535 W" and
"474 raw files". The witness maximum is 538.03 W and 23 samples stand above 535; the
archive holds 520 raw files, all written by 12:07, with no later addition to explain a gap of
46; and 21.2 GiB matches none of the three weight footprints this session actually printed
(19.6 / 20.0 / 20.4 GiB). The old numbers are printed here rather than dropped.

Observation

resolve_sparse_moe_small_t_plan carries two arms of the same main-profile ladder that differ in one
number: the Q5 arm gives up the Rows2 D4 schedule above five tokens, the Q6 arm holds it to eleven
(src/ops/sparse_moe/small_t/sparse_moe_small_t_plan.cpp:46-54 on this base, with the shared
plan.d3_schedule assignment on :45). The Rows parameter
decides only how many destination rows one routed-weight stream serves; the rank-order FP32 epilogue
is identical either way.

A codec-dependent threshold is defensible in principle - Q6 reads more weight bytes per row, so it
tolerates the extra activation re-reads of the narrower schedule for longer - and it is not a general
rule in this op either way: the sibling prefill plan gives both codecs the same entry point
(kSparseMoePrefillQ4Q5Min == kSparseMoePrefillQ4Q6Min == 47,
src/ops/sparse_moe/prefill/sparse_moe_prefill.h:17-18).

What I have is a measurement of the crossover itself, on both codecs, with the schedule forced.
On a16b6442 it lands at T=19 on Q5 and T=17 on Q6: two tokens apart, where the ladder separates
the arms by six. A codec-dependent threshold is real here, but it is a third of the one that is
written down, and both crossovers sit far above both constants in the tree.

37 of the 40 MoE layers on this artifact take the Q5 arm. That is the loader's own rule rather than
a property of one file: routed_down_format returns Q6G64_F16S for layers 34, 38 and 39 and
Q5G64_F16S for every other layer
(src/targets/qwen3_6_35b_a3b/impl/load/bindings.cpp:20-23), against 40 layers in total - 10 full
attention plus 30 GDN (src/targets/qwen3_6_35b_a3b/impl/config.h:65-66). So the arm this change
moves furthest is the one that carries 37 of 40 layers.

Change

One file, src/ops/sparse_moe/small_t/sparse_moe_small_t_plan.cpp, 8 insertions and 9 deletions,
of which 5 insertions are the comment recording where the constant comes from. The routed_down
branch is deleted and one ladder replaces both arms:

    plan.d4_schedule = tokens <= 2 ? SparseMoeSmallTD4Schedule::Rows1
                       : tokens <= (routed_down == QType::Q5G64_F16S ? 19 : 17)
                           ? SparseMoeSmallTD4Schedule::Rows2
                           : SparseMoeSmallTD4Schedule::Rows4;

Effect per arm: Q5 goes from 5 to 19, Q6 from 11 to 17. T <= 2 keeps Rows1 on both, as
before. Each arm now carries the value its own forced-schedule sweep produced, and the branch stays
because the two values differ.

Superseded. The previous revision of this PR collapsed the branch and gave both codecs the
single constant 19, on the ground that the admissible intervals [19,25] and [17,20]
intersect. That intersection exists only under the reading that takes the contested Q6 cells at
T=18 and T=19 as lattice noise - the one contest in this report resolved in the change's favour.
Codex pointed out that the same report puts the Q6 crossover at 17 outright, so a single constant
of 19 bets three Q6 layers on that reading. It is the right objection: 17 is no worse than 19
for Q6 under either reading of those cells, and better under one of them, so the trade is
unnecessary and this revision does not take it. The Q5 answer is unchanged.

Affected behavior and ownership boundary

No public contract changes. No CLI, workspace, graph-profile or artifact change. The op's numerical
contract is untouched, and that is demonstrated bitwise below rather than asserted.
sparse_moe_small_t_workspace_bytes depends on tokens, not on the schedule, so the reported
workspace is identical at every width on both arms.

The plan has exactly two consumers, both in src/ops/wrapper/sparse_moe.cpp (:221 workspace,
:246 launch); resolve_sparse_moe_small_t_plan is called nowhere else in src/, tests/ or
bench/. The adaptive (persistent-grid) route does not go through the plan at all:
src/ops/sparse_moe/prefill/sparse_moe_prefill_kernels.cu:1218 calls
sparse_moe_decode_launch_d4_small_t with a hard-coded SparseMoeSmallTD4Schedule::Rows4 (:1220).
That matters, because the adaptive grid is kAdaptiveD4Blocks = 5 * (kHidden / 4)
(src/ops/sparse_moe/decode/sparse_moe_decode_kernels.cu:31), a constant derived from Rows4; if
the adaptive path read the plan, this change would move a persistent grid. It does not.

The change is live in two places, not one: in a speculative or batched decode round of 6..19
columns, and in a prefill chunk of 2..19 tokens - which, at the default prefill_chunk of 1024,
means any prompt of nineteen tokens or fewer, and the tail chunk of any longer prompt whose remainder
falls there. The next section reads that out of the source. Neither end-to-end ladder in this report
exercises the second one, and that is stated again under Limitations rather than left implicit.

Which widths the product can actually ask for

This decides what the constant is worth, so it is read out of the source rather than out of memory.
The operator has two entry points, and an earlier draft of this section counted only the first.

Entry 1, a decode round. The width the small-T operator is called with is the number of
columns in the batch, not the draft window:

src/targets/qwen3_6/impl/runtime/text_context_impl.h:699   const std::int32_t width = ids.ne[0];
src/targets/qwen3_6/impl/runtime/text_context_impl.h:700   const std::int32_t batch = ids.ne[1];
src/targets/qwen3_6/impl/runtime/text_context_impl.h:705   const std::int32_t columns = width * batch;

and columns tokens then travel run_layers -> Variant::post_mixer
(src/targets/qwen3_6_35b_a3b/impl/variant.cpp:218) -> run_sparse_moe (:66-73) -> ops::sparse_moe,
where tokens = columns reaches resolve_sparse_moe_small_t_plan
(src/ops/wrapper/sparse_moe.cpp:246).

Entry 2, a prefill chunk. The route is chosen from the token count alone, with no notion of
phase. src/ops/wrapper/sparse_moe.cpp:208-210 computes use_small_t = sparse_moe_uses_small_t(tokens) and use_prefill = sparse_moe_uses_prefill(tokens, ...); the
prefill route opens only at tokens >= minimum and that minimum is 47 for both codecs
(src/ops/sparse_moe/prefill/sparse_moe_prefill_plan.cpp:24-28, with
kSparseMoePrefillQ4Q5Min == kSparseMoePrefillQ4Q6Min == 47), while the small-T domain is [2,46]
(sparse_moe_small_t_plan.cpp:9-11). So every call into ops::sparse_moe with tokens in
[2,46] lands in the ladder this PR edits, whichever phase issued it - and prefill issues such
calls. A prefill chunk is

src/targets/qwen3_6/impl/runtime/text_context_impl.h:1107   int len = std::min(chunk, T - t0);

(further clipped by the split frontier on :1108), and that chunk then runs
run_layers(x, Phase::Prefill, tap) (:1184) -> mlp_tail (:1018, :1039) ->
Variant::post_mixer (:993) -> src/targets/qwen3_6_35b_a3b/impl/variant.cpp:218 ->
run_sparse_moe with hidden.ne[1] = len (variant.cpp:66-73). The default prefill_chunk is
1024 (include/ninfer/types.h:160, apps/cli/options.h:24, and the value in the prefill_chunk
column of all eight ladder CSVs), so a prompt shorter than 1024 tokens is one chunk of its own
length, and the tail chunk of any prompt is the remainder - any number from 1 to chunk.

quantity	value	reference
kMaximumConcurrency, the cap on batch	8	include/ninfer/types.h:20
kMtpDecodeMaximumDrafts	5, so width <= 6	src/targets/qwen3_6/export/ninfer/targets/qwen3_6/round_state.h:15-16
kDFlashDecodeMaximumDrafts	15, so width <= 16	same file, :17-18
--spec mtp requires --draft-tokens in [1,5]		src/product/speculative_options.h:41
--spec dflash requires [1,15]		:46
--spec dflash2 requires [1,15]		:51
ordinary decode: width is bound to 1, batch to [1,8]	T = batch	src/targets/qwen3_6/impl/runtime/text_context_impl.h:652-654,679
small-T domain	[2, 46]	src/ops/sparse_moe/small_t/sparse_moe_small_t.h:14,16
above it, the prefill route	T >= 47, both codecs	src/ops/sparse_moe/prefill/sparse_moe_prefill.h:17-18

Inside entry 1 the reachable set is { width x batch }:

backend	reachable T in [2,46]
ordinary decode (width = 1)	2..8
--spec mtp (width = 2..6)	2,3,4,5,6,8,9,10,12,14,15,16,18,20,21,24,25,28,30,32,35,36,40,42
--spec dflash / dflash2 (width = 2..16)	2..16, 18,20,21,22,24,25,26,27,28,30,32,33,35,36,39,40,42,44,45

Eleven values in [2,46] are not a product of width <= 16 and batch <= 8 - 17, 19, 23, 29,
31, 34, 37, 38, 41, 43, 46: the eight primes above 16, plus 34 = 2 x 17, 38 = 2 x 19 and
46 = 2 x 23, whose only factorisations need a width above 16 - so no decode round can ask for
them. Entry 2 reaches all of them. A prompt of nineteen tokens, or any prompt whose tail chunk is
nineteen tokens, enters the ladder at T = 19. The reachable set of this operator is therefore the
whole of [2,46], 18 and 19 are distinguishable in the product, and the constant proposed here
is a width the product can actually ask for.

Corrected on a16b6442, 2026-09-07 - two corrections, the second one to this package's own
draft.

(a) The previous revision of this text, and the 2026-09-03 sweep notes behind it, said:
"Those widths are reachable only through --spec dflash; --spec mtp caps at --draft-tokens 5,
i.e. T=6." That is true only at batch 1; the correct statement is the factorisation above. The
old wording is quoted because it is what the earlier package claimed - but only half of it was
still standing when this revision began. The published body already withdrew the other half: the
allocation gate is is_masked_draft_backend(layout.spec.backend)
(src/targets/qwen3_6/impl/state/round_state.cpp:174), which is
backend == SpeculativeBackend::DFlash || backend == SpeculativeBackend::DFlash2
(.../qwen3_6/startup_features.h:7-9), so the route is opened by --spec dflash2 as well, and
that package stated the "reachable only through --spec dflash" wording "have been corrected
wherever they appeared". This block supersedes that one; it does not re-open it.

(b) An intermediate draft of this revision then wrote: "At batch = 8, mtp reaches
T=42 and dflash reaches T=45", and drew from it: "T=17 and T=19 are unreachable at
any batch size, so thresholds 18 and 19 are indistinguishable in the product". Both are wrong,
and both are wrong in this PR's own favour, which is why they are written out rather than edited
away. At a batch of exactly 8, width x 8 inside [2,46] is {16, 24, 32, 40} for either
backend; 42 is 6 x 7 and 45 is 15 x 3, so the true statement is "with a batch of up to
eight", not "at a batch of eight". And "unreachable at any batch size" is a statement about
decode columns only - prefill reaches every width in [2,46], T=19 included. 18 and 19 are
distinguishable, and the case for 19 does not rest on their being interchangeable: it rests on 19
being the measured Q5 optimum at a reachable width.

Crossover sweep, on a16b6442

Two extra builds of the operator benchmark rewrite both arms of the ladder at once
(resolve_sparse_moe_small_t_plan patched by script): one forces Rows2 at every width in [3,46],
the other forces Rows4. Rows1 at T <= 2 is kept on both, so T=2 is a built-in null cell and
reads +0.00% in every pass on both codecs. Rows is only a grid split
(grid = dim3(kHidden / Rows, tokens), sparse_moe_decode_kernels.cu:619), both instantiations
already exist for both codecs on master, and the workspace does not depend on the schedule, so
forcing either one is legal across the whole range.

delta = (Rows2 - Rows4) / Rows4; negative means Rows2 is faster. * marks full sample separation
(the worst pass of the winner beats the best pass of the loser). Five sweep windows plus single-point
--tokens T runs, ABAB alternation, 5 scoring passes per arm per window (3 for the single points):

Q5G64_F16S

T	M 2:30	F 10:24	H 12:30	K 16:30	G 16:46	single point	verdict
3	-8.33*						Rows2
6	-2.94*						Rows2
8	-4.88*						Rows2
10	-4.17*	-4.17*					Rows2
11	-1.96*	-1.96*				-1.96*	Rows2 - the threshold this PR used to propose
12	-1.82*	-1.82*	-1.82*			-1.82*	Rows2
13	-1.69*	-1.69*	-1.69*			-1.69*	Rows2
14	-3.15	-1.59*	-1.59*			-1.59*	Rows2
15	-1.52*	-1.52*	-1.52*			-1.52*	Rows2
16	-1.45*	-1.43	-0.02	-1.45*	-1.45*	-1.45*	Rows2 - the 2026-09-03 "optimum"
17	-1.37*	-1.37*	-1.37*	-1.37*	-1.37*	-1.37*	Rows2
18	-1.30*	-1.30*	-1.30*	-1.30*	-1.30*	-1.30*	Rows2
19	-1.23*	-1.23*	-2.44*	-1.25*	-1.25*	-1.23*	Rows2 - measured optimum
20	+0.00	-1.18*	-1.18*	-1.18*	-1.18*	+0.00	crossover, taken as zero
21..25	+0.00	+0.00	+0.00 / -1.08* / -1.03*	+0.00 / +0.96	+0.00 / +0.96	-0.02..+0.00	indistinguishable
26	+0.00		+0.93	+0.00	+0.91	+0.92	contested
30	+0.81*		+0.79	+0.81*	+0.81*		Rows4
31..46					+2.31..+5.07*		Rows4

Q6G64_F16S

T	M 2:30	F 10:24	H 12:30	K 16:30	G 16:46	single point	verdict
3..10	-2.04..-6.90*						Rows2
11	-3.77*	-3.77*				-3.77*	Rows2 - the constant in the tree today
12	-3.51*	-3.51*	-3.51*			-3.51*	Rows2
13	-1.67*	-1.67*	-1.67*			-1.67*	Rows2
14	-3.08*	-3.08*	-3.08*			-3.08*	Rows2
15	-1.47*	-2.90*	-2.90*			-1.49*	Rows2
16	-1.39*	-2.74*	-2.74*	-1.39*	-1.39*	-1.39*	Rows2
17	-1.33*	-1.33*	-0.02	-1.33*	-1.33*	-1.33*	Rows2 - measured optimum
18	+0.00	+1.27*	+1.27*	+1.25	+0.02	+0.00	crossover, taken as zero
19	+0.00	+1.20*	+0.02	+1.19	+1.19	+0.00	taken as zero
20	+0.00	+0.00	+0.00	+0.00	+0.00	+0.00	indistinguishable
21	+1.10*	+1.10*	+1.10*	+1.10*	+1.10*	+1.10*	Rows4, first firm win
22	+1.06*	+1.06*	+2.13*	+1.06*	+1.06*	+1.06*	Rows4
23	+2.02*	+2.02*	+2.02*	+2.02*	+2.02*	+2.02*	Rows4
24..46					+1.94..+5.25*	+1.94..+3.64* (T=24..27)	Rows4

Why five windows, and how the contested cells were settled

Readings lie on a 2.048 us lattice; at T=20 one step is 1.18% of the operator. Raising --repeat
does not help - within an arm at --repeat 400 the spread is 0.00% and the cell can still sit on the
wrong step. Contested cells were settled by changing the sweep window and by single-point runs.
Both contests are recorded rather than rounded away, and they did not resolve the same way.

Corrected. An earlier revision of this section said "both were resolved against the
change". That is true of the Q5 contest and false of the Q6 one. Taking the Q6 cells as zero is
what leaves the top of its indifference band at 20 and the shared constant free; reading them as
signal makes 19 an explicit trade, as the paragraph below already priced. The Q6 contest was
resolved in the change's favour, and calling it otherwise flattered the result.

• Q5, T=20. Four of five windows read -1.18% with full separation; window M and the single point
  read exactly +0.00%. On the lattice the cause is visible: the Rows2 arm reads 172.032 us in
  every configuration, while the Rows4 arm reads 174.080 us when the sweep starts at 10/12/16 and
  172.032 us when it starts at 2 or is measured as a single point. The losing arm is what moves,
  and sweep order moves it, not the schedule. Taken as zero, so the Q5 optimum is 19. Believing
  the four windows would have made it 20, which is the constant this change would then have
  proposed instead of 19 - not a difference that stops at this cell.
• Q6, T=18 and T=19. This is the weakest reading in the report, and it is described here the way
  the table above prints it. At each of the two widths three of the five windows read non-zero
  for Rows4, and they are not the same three: at T=18 it is F (+1.27%), H (+1.27%) and K
  (+1.25%), at T=19 it is F (+1.20%), K (+1.19%) and G (+1.19%). Only three of those six
  readings carry full sample separation - F and H at T=18 (raw/200_analiz_blokov.txt:141, :223)
  and F at T=19 (:142); in K and G at both widths the samples overlap
  (raw/200_analiz_tochek.txt:82-83 for K, raw/200_analiz_blokov.txt:279-280 for G).
  Which arm moves is not the same in all four of those cells, and the exception is the one that
  argues against this change, so it is printed rather than summarised. In three of them the Rows2
  arm moves by one lattice step while Rows4 stands still: K at T=18 (Rows2 163.808 at 1.27%
  spread against Rows4 161.792 at 0.00%), G at T=18 (161.824 at 1.27% against 161.792 at 0.00%)
  and G at T=19 (172.032 at 1.19% against 170.016 at 0.02%). In the fourth, K at T=19, it is the
  other way round: Rows2 reads 172.032 in every pass and Rows4 is the arm that moves (170.016,
  1.19% spread), with four of its five passes a whole lattice step below the stationary arm.
  That is the strongest single reading in this report for Rows4 winning at T=19, and it sits
  inside the contest this section resolves the other way. Window M
  (raw/200_analiz_blokov.txt:95-96) and the single point (raw/200_analiz_tochek.txt:30-31) read
  +0.00% at both widths, and one window per width reads a bare timer tick - G at T=18 (+0.02%) and
  H at T=19 (+0.02%). Taken as zero, so the top of the Q6 indifference band is 20 and the first
  firm Rows4 win is 21, where all five windows and the single point agree with full separation.
  What that decision is worth if it is wrong: about 1.2% of the operator at T=18 and T=19 on the
  Q6 arm, i.e. on 3 of the 40 MoE layers, at widths the product can reach through either entry point.
  Reading it the other way would put the Q6 optimum at 17 and shrink the Q6 admissible interval to
  [17,17], which does not intersect Q5's [19,25] at all: there would then be no free shared
  constant, and 19 would become an explicit trade - about 1.2% given up at T=18 and T=19 on the three
  Q6 layers against 1.23% to 1.30% gained at the same widths on the thirty-seven Q5 layers. The Q5
  answer does not move either way. That dependency is real and is stated here rather than buried.
• One cell that must not be sold as signal: Q5 T=21 reads -0.02% at the single point with formal full
  separation. That is 0.032 us, one timer tick, not a lattice step. Zero.

Null control and noise floor

• Null control, blocks Z and Z2: 37 of 40 cells exactly +0.00%, three at +0.02%. The
  forced-Rows4 arm is run against a byte-identical copy of itself (same md5) through the same
  ABAB pipeline. Block Z, over the 10:24 window, is 30 cells; 28 read exactly +0.00%, and the
  two that do not are q4-q5 at T=19 and q4-q6 at T=22, both +0.02%
  (raw/200_analiz_blokov.txt:162 and :185). Block Z2, the same on single points at
  T = 17,18,19,20,23, is 10 cells; 9 read exactly +0.00%, and the one that does not is q4-q5
  at T=19, +0.02% (raw/200_analiz_tochek.txt:47). All three are +0.02% = 0.032 us, one
  timer tick, which is a sixty-fourth of the 2.048 us lattice step, and all three have
  overlapping samples, which is the property the decision rule keys on. On the 2026-09-03 session
  this control had 28 of 30 clean with two one-lattice-step false cells at T >= 23; this session
  has no false cell of that size, which is the comparison that matters - but it is not a clean
  40 of 40 and the report should not say so. Operationally: a reading with full sample separation and
  a whole lattice step behind it is signal; a reading of +-0.02% is a timer tick and is zero.

  Where this bites, stated at its worst rather than its best. Two of the three tick cells
  sit at q4-q5 T=19 - the width this PR proposes as the constant - block Z at
  raw/200_analiz_blokov.txt:162 and the single point at raw/200_analiz_tochek.txt:47; the third
  is q4-q6 T=22 (:185). Both T=19 cells are printed in the package's own summary,
  raw/200_svodka.txt, in its rightmost null-control column.

  And +-0.02% is the floor of the median delta, not of the cell. The block Z cell at T=19
  reads 167.936 against 167.904 with spreads of 1.23% and 1.22%, so read conservatively -
  worst pass of one arm against best of the other - the same null cell is +1.22%, a whole
  lattice step, at the constant this change proposes. The single point at that width is tight
  (0.02% on both arms), and the two disagree.

  It does not move the answer: the signal at that cell is -1.23% with full sample separation in
  five windows and at the single point, and a null control that can read one step under the
  conservative rule is exactly why the rule used here is separation plus a step, which this cell
  fails on separation. But the honest summary is that the noise floor of this instrument is one
  lattice step under conservative reading and a timer tick under median reading, and that both
  numbers land on T=19.

  Corrected before submission. An earlier draft of this bullet read: "30 of 30 cells read
  exactly +0.00% ... Block Z2 ... 10 of 10 at +0.00% ... this session has none [false
  cell]", and two sentences further down carried "with the null control at 40 of 40 cells on
  zero". The attached raw prints +0.02% in three of those forty cells, named above with file and
  line, and two of the three are at the proposed constant. 37 of 40 is what was measured. This
  is the same class of error as the Corrected block further down about "exactly +0.00% at all
  eleven widths" - a control rounded up to perfect - so it is written out in full here rather than
  repaired quietly.

• Noise floor, block N, five passes of one arm back to back: median (max-min)/min is 0.00% on
  all four series; the per-cell maxima are 1.19..2.86%, and every one of those is exactly one
  2.048 us lattice step.

Did the 16/17 crossover hold

Q6: exactly. 17 then, 17 now.

Q5: no. The optimum moved from 16 to 19. The difference is not subtle. On a140e7ae the cells
Q5 T=17..19 read -0.02..-0.01%, i.e. zero. On a16b6442 the same three cells read
-1.37 / -1.30 / -1.23% with full separation in five windows and at the single point, against
a null control that is exactly on zero in 37 of 40 cells and one 0.032 us timer tick off it in the
other three. (T=20, the fourth width of the old interval, is the contested cell discussed above and
is taken as zero on this base; an earlier draft of this sentence wrote "T=17..20" over three
numbers.) Between the two bases master rewrote
src/ops/sparse_moe/small_t/sparse_moe_small_t_kernels.cu (102 lines) and
src/ops/sparse_moe/sparse_moe_route.cuh (70 lines) - a plausible cause, but not a checked one; I
took no per-commit bisect of the shift.

Superseded on a16b6442, 2026-09-07. The previous revision of this text said: "The measured
optimum is 16 for Q5 and 17 for Q6. Because a plateau follows each, any threshold in [16, 30] is
optimal for Q5 and any threshold in [17, 22] is optimal for Q6. Those intervals intersect, so a
single shared constant is available without a compromise, and its best value is 17: exactly optimal
for Q6, and free for Q5, whose T=17 measures -0.02%, one 0.032 us tick." For Q6 that
still holds. For Q5 it does not hold on a16b6442: the optimum is 19, the admissible
interval is [19, 25], the intersection with Q6's [17, 20] is [19, 20], and the best
shared value is 19. The old sentence is kept verbatim rather than deleted; it is correct for
a140e7ae, the base it was taken on.

The three numbers that follow

	Q5 (Q5G64_F16S)	Q6 (Q6G64_F16S)
in the tree today, a16b6442	5	11
proposed by the previous revision of this PR	11	11 (untouched)
measured 2026-09-03 on a140e7ae	16	17
measured 2026-09-07 on a16b6442	19	17
indifference band above the optimum	20..25 (26 contested)	18..20
first firm Rows4 win	30 (26 and 28 contested)	21
admissible threshold interval	[19, 25]	[17, 20]

The intervals intersect at [19, 20], so a single shared constant is arithmetically available.
This PR does not take it, and the reason is below.

Why 19 for Q5, 17 for Q6, and why the branch stays

1. Each arm takes the value its own sweep produced. Q5's optimum is 19, Q6's is 17, both measured
   on this base with a forced schedule rather than inherited from the neighbouring codec. Neither
   number rests on a judgement call.

2. The shared constant would have rested on one. 19 sits inside Q6's admissible interval only
   because T=18, 19 and 20 are taken as zero there: three of five windows at each of T=18 and
   T=19 read about +1.2% for Rows4, three of those six readings with full sample separation, and
   the null control is exactly on zero in 37 of 40 cells rather than in all 40. Under the other
   reading Q6's interval is [17,17], the intersection is empty, and 19 costs about 1.2% on the
   three Q6 layers.

3. 17 is not worse than 19 for Q6 under either reading, and better under one. If the contested
   cells are lattice noise, T=18 and T=19 run the same either way and the choice is free; if they
   are signal, 17 is right and 19 is a regression. There is no reading in which 19 wins on Q6, so
   there is nothing to buy by collapsing the branch - only a bet to place.

   Superseded. The previous revision of this PR took that bet: one constant, 19, branch
   removed, justified by the interval intersection. Codex objected that the report's own Q6
   crossover is 17 and that collapsing the branch therefore introduces a measured Q6 regression at
   widths the product can reach. The objection is correct and this revision drops the shared
   constant. What the earlier revision got right and is kept: the branch as it stood on master was
   not supported by measurement either - it separated the codecs by six tokens where the sweep
   separates them by two.

4. The branch is no longer degenerate. Under the previous revision of this PR both if arms
   were byte-identical, so the condition selected nothing. With 19 and 17 it selects again, and the
   two tokens between them are what the measurement actually found.

5. 19 rather than 18 or 20. 19 is the measured Q5 optimum and a width the product can ask
   for: a nineteen-token prompt, or any prompt whose tail chunk is nineteen tokens, enters the
   ladder at T=19 (see Which widths the product can actually ask for). 18 would give T=19 away on
   Q5 - -1.23% of the operator, full separation in five windows and at the single point. 20 is
   admissible too, but it is one width past the last unambiguous Q5 win and would plant the constant
   on the cell this report resolved against itself. Of the admissible numbers, the one standing
   exactly on the measurement is taken.

6. Not 17 for Q5, the answer of the old sweep. 17 leaves T=18 on the table for Q5: -1.30% of the
   operator, full separation in five windows and at the single point, and T=18 is reachable -
   width=9, batch=2, width=6, batch=3, width=3, batch=6, all inside width <= 16, batch <= 8,
   and also as an eighteen-token prefill chunk.

What stopping at eleven would have cost

Single points, pass 0 discarded, median of three, on a16b6442:

• Q5, T=12..19: 1.23% to 1.82% of the operator forgone. The bot's estimate was "1.47-1.82% at
  T=12..16"; on the current head the band is wider and runs to T=19, not to T=16.
• Q6, T=12..17: 1.33% to 3.51% of the operator forgone.

Roofline

MMA: zero, and that is a property of the kernel, not an omission. The D4 small-T kernel carries no
MMA tier at all - dot_fp32_rows
(src/ops/sparse_moe/decode/sparse_moe_decode_kernels.cu:325) is a SIMT FP32 FMA loop over the
row-split codec followed by a warp reduction, and the translation unit contains no mma/wmma
instruction (re-checked on a16b6442: zero matches). So the share of the MMA peak is 0% on both arms
at every width, and the only ceiling that applies is memory.

GDDR7: 44-48% of the vendor peak on both arms, everywhere Rows2 wins. The numerator is the
benchmark's own unique_weight column, median of block K over its five scoring passes per arm. The
denominator is theoretical_memory = 1792.1 GB/s, the figure the benchmark prints in the header of
every raw file, and the percentage in each cell below is the one the benchmark itself prints
beside that reading - not one computed here:

T	Q5 Rows2	Q5 Rows4	Q6 Rows2	Q6 Rows4
16	846.9 GB/s = 47.3%	834.6 = 46.6%	867.9 = 48.4%	855.8 = 47.8%
17	824.2 = 46.0%	813.0 = 45.4%	858.2 = 47.9%	846.7 = 47.2%
18	827.2 = 46.2%	816.4 = 45.6%	841.2 = 46.9%	851.7 = 47.5%
19	818.8 = 45.7%	808.6 = 45.1%	834.7 = 46.6%	844.6 = 47.1%
20	811.3 = 45.3%	801.7 = 44.7%	838.5 = 46.8%	838.5 = 46.8%
21	785.5 = 43.8%	785.5 = 43.8%	813.4 = 45.4%	822.4 = 45.9%

Corrected before submission. An earlier draft of this table carried the same GB/s against a
different denominator: "The denominator is the read ceiling measured on this card, 1689.4
GB/s, not the 1792.1 GB/s the benchmark header prints from theoretical_memory_gbps (that one is
~6% more optimistic and is used nowhere here)", which turned the same readings into 46.5-51.4%.
The string 1689 appears in no file of this campaign. It is a number from an earlier session
of ours, and the read-bandwidth probe behind it was not preserved - two other packages from the
same working tree state exactly that and decline, for that reason, to quote any roofline
percentage at all. Quoting it here as "measured on this card" would have presented a carried
number as a fresh one. The denominator is therefore the vendor peak the tool prints, and the
percentages are the tool's own; the GB/s column is unchanged except for two cells that moved by one
tenth on recount of the block-K median - Q5 Rows4 at T=20 is 801.7, not 801.8, and Q6 Rows2
at T=18 is 841.2, not 841.1.

Both arms sit between 44% and 48% of the vendor peak across the whole interval where Rows2 wins
and slide down towards T=21. That is the profile of a latency-limited kernel, not a
bandwidth-limited one: the narrower schedule buys latency hiding, not bandwidth. Which also means
the roofline does not explain where the crossover sits - see Limitations.

Verification, taken at the constant 19 on a16b6442

Candidate arm: a16b6442 plus the single constant 19 (cli_cand, md5 c7538e06...). Master arm:
bare a16b6442 (cli_master, md5 8d74557f...).

Bitwise gate: 24 of 24 IDENTICAL

ninfer <model> --prompt <text> --max-context 8192 --max-new 160 --greedy --seed 1 --raw-output [--spec ...]

Three prompts (short / code / math) x four modes x two repetitions. On rep1 the arm order is
master, cand, master2; on rep2 it is reversed. Each cell is its own single-shot process under the
lock. A cell is declared INVALID, not equal, if either arm produced under 40 bytes - the CLI default
context is 2048 and a long prompt can drive both arms to empty output, where a naive MD5 comparison
of two empty files always passes; --max-context 8192 is set for that reason.

mode	T per round	cand vs master	master2 control
no speculation	1	6/6 IDENTICAL	6/6 IDENTICAL
--spec mtp --draft-tokens 5	6	6/6 IDENTICAL	6/6 IDENTICAL
--spec dflash --draft-tokens 11	12	6/6 IDENTICAL	6/6 IDENTICAL
--spec dflash --draft-tokens 15	16	6/6 IDENTICAL	6/6 IDENTICAL
total		24/24, 0 invalid	24/24

Both ends of the T=12..16 band now have end-to-end coverage - T=12 and T=16, through
--spec dflash rather than through mtp - which is exactly what the previous revision of this PR
did not have and named as a gap. T=13, 14 and 15 were not run: the gate steps by the draft
window, and no intermediate --draft-tokens value was measured. An earlier draft of this sentence
said the band was "covered end to end", which reads as coverage of all five widths.

What the gate does and does not exercise on the prefill side. All three prompts are short enough
to enter the small-T route in prefill as well: the engine's own summary reads prompt tokens 27 for
the short and math prompts and 25 for the code prompt, identically in all 78 recorded runs
(gate/err_*.txt), and both counts are inside the operator's [2,46] domain, so every cell pushed
one prefill chunk through this ladder. Both counts are above 19, so both arms select Rows4
there and the change is inert at that width: the gate demonstrates that the prefill entry is real and
exercised, not that the constant is bitwise safe at a prefill width where it acts. No run in this
package uses a prompt of nineteen tokens or fewer.

Gate strength control, and it is not clean. A sabotage arm was built: the D4 small-T epilogue
multiplied by 1.001 (sparse_moe_decode_kernels.cu, the line
*output = __float2bfloat16_rn(value)), md5 466ba764.... Six cells (mtp5 and dflash15 x three
prompts): 5 of 6 DIFFER, one - dflash15/math - IDENTICAL. The gate can tell arms apart, but it
let a 0.1% numeric corruption through in one cell of six. That is the known blindness of a bitwise
gate, and it is stated here rather than left out.

The master2 control is a byte-identical rebuild of cli_master (same md5, the build is
reproducible), not an independent one. As a null control for the pipeline it is sound; as evidence of
independence it is not.

End-to-end ladder

ninfer_bench, -pg 128,128 -pg 4096,128 -r 3 --warmup 1 --max-ctx 8192, ABAB alternation, 4 passes
per arm, pass 0 discarded, median of three.

mode	T	ladder	prefill tok/s master -> cand	decode tok/s master -> cand	decode delta
no speculation	1	pp128+tg128	6362.3 -> 6351.5 (-0.17%)	372.8 -> 373.3	+0.13%
no speculation	1	pp4096+tg128	19075.4 -> 19076.3 (+0.00%)	363.1 -> 362.9	-0.07%
mtp --draft-tokens 5	6	pp128+tg128	5779.2 -> 5777.9 (-0.02%)	300.6 -> 304.8	+1.38%
mtp --draft-tokens 5	6	pp4096+tg128	18105.8 -> 18108.5 (+0.02%)	797.7 -> 808.5	+1.37%
dflash --draft-tokens 11	12	pp128+tg128	6206.4 -> 6218.3 (+0.19%)	168.1 -> 169.6	+0.89%
dflash --draft-tokens 11	12	pp4096+tg128	18065.7 -> 18066.5 (+0.00%)	1038.5 -> 1045.4	+0.67%
dflash --draft-tokens 15	16	pp128+tg128	6207.0 -> 6199.9 (-0.11%)	147.8 -> 148.5	+0.44%
dflash --draft-tokens 15	16	pp4096+tg128	18170.1 -> 18144.9 (-0.14%)	1276.9 -> 1282.4	+0.43%

Two checks inside the same table:

• The no-speculation row is the internal control. At T=1 the change cannot act, and it does not:
  +0.13% and -0.07%.
• Draft acceptance matches to the third decimal on both arms in all six speculative cells
  (accept m=0.190 c=0.190, 0.881/0.881, 0.028/0.028, 0.734/0.734, 0.025/0.025,
  0.814/0.814) - independent evidence of bitwise equality: the stream of acceptance decisions did
  not change.

Round step

step = decode_seconds_mean * 1000 / spec_rounds, derived from the same ladder CSVs. This is a
derived quantity, not a separate instrument - no dedicated graph-step capture was taken this
session.

mode	T	ladder	master, ms	cand, ms	delta
mtp 5	6	pp128+tg128	2.1505	2.1212	-1.36%
mtp 5	6	pp4096+tg128	2.2287	2.1987	-1.35%
dflash 11	12	pp128+tg128	2.5902	2.5674	-0.88%
dflash 11	12	pp4096+tg128	2.9346	2.9152	-0.66%
dflash 15	16	pp128+tg128	3.0704	3.0571	-0.43%
dflash 15	16	pp4096+tg128	3.3414	3.3270	-0.43%

Because the round count is identical across the arms (spec_rounds and acceptance match exactly),
the whole end-to-end figure is round speed, not a substitution of round count. See the last section.

ctest: a targeted subset, 6 of 6 on both arms - and the full suite was not run

The full ctest was not taken, and the reason is disk, not a decision. Building the 110 test
targets did not fit: the build filled the root filesystem and was stopped. The figure the session
recorded for the object tree is 14 GB, written into the header of the script that replaced the full
run (ctest2.sh:2); the df lines that survive in the logs are from after the recovery and read
13G and 14G free of 152G at 92% and 91% (logs/ctest2_out.txt, logs/buildv_out.txt). My own
build/tests directory was deleted and 13-14 GB were recovered. Nothing belonging to anyone else was
touched.

Corrected before submission. An earlier draft of this paragraph said the disk "hit 100% with
49 MB free". That number is in no file of this session; what the session preserved is the
14 GB object figure in the script header and the post-recovery df lines above. The exact free
space at the moment the build was stopped was not captured, so it is no longer claimed.

Instead, the tests this change can plausibly reach were built and run on both arms out of one build
directory, sequentially, because two trees plus the artifact do not fit on this disk:

test	cand	master
ninfer_public_api_test	Passed	Passed
ninfer_decode_graph_test	Passed	Passed
ninfer_qwen3_6_35b_a3b_real_test	Passed, 40.94 s	Passed
ninfer_qwen3_6_35b_a3b_dflash_real_test	Passed, 70.03 s	Passed
ninfer_qwen3_6_35b_a3b_dflash_load_plan_test	Passed, 0.02 s	Passed
ninfer_sparse_moe_test	Passed, 6.86 s	Passed, 6.95 s
total	100% of 6, 0 skipped	100% of 6, 0 skipped

The first run gave 3 Passed and 3 Skipped (logs/ctest_cand.log): ninfer_qwen3_6_35b_a3b_real_test,
ninfer_qwen3_6_35b_a3b_dflash_real_test and ninfer_qwen3_6_35b_a3b_dflash_load_plan_test all
carry SKIP_RETURN_CODE 77 and skip without NINFER_QWEN3_6_35B_A3B_WEIGHTS
(tests/CMakeLists.txt:165, :169 and :178-179). Re-run with that variable pointing at the real
artifact, no skips remain. An earlier draft cited only :165,169 and described the three as "the
*_real_* tests"; the third one is ..._dflash_load_plan_test, which does not match that pattern
and carries its skip code on :178-179.

What this does not replace. The other 108 registered tests were run on neither arm at this
constant. The diff touches one .cpp and does not touch tests/CMakeLists.txt, so the registered set
does not change - but that is a prediction, not a measurement.

clang-format

Re-run on a16b6442 against this diff, since removing the branch re-indents the hunk:
clang-format --dry-run -Werror src/ops/sparse_moe/small_t/sparse_moe_small_t_plan.cpp exits 0 with
no diagnostics, on clang-format 21.1.8. The repository pins SortIncludes: Never
(.clang-format:39), and this diff adds no include.

What the earlier session measured, and on which diff

Kept for the record, not relabelled. Everything in this section was taken on a140e7ae /
ad0f3d38, against a diff that set the Q5 threshold to eleven and left Q6 alone. It is not
evidence about the diff proposed here.

Operator, on a140e7ae, two independent passes per arm with the arm order reversed on the
second; each delta taken from the faster master pass against the slower change pass:

T	master us, pass 1 / 2	with change, pass 1 / 2	delta
2	38.880 / 38.880	38.880 / 38.880	+0.00%
3	45.024 / 45.056	45.056 / 45.056	+0.07%
4	53.216 / 53.216	53.248 / 53.216	+0.06%
5	59.392 / 59.392	59.392 / 59.392	+0.00%
6	71.424 / 69.664	67.584 / 67.584	-2.99%
7	75.808 / 75.776	73.728 / 73.728	-2.70%
8	83.968 / 83.968	79.872 / 79.872	-4.88%
9	90.112 / 90.112	86.048 / 86.048	-4.51%
10	98.304 / 98.304	94.240 / 94.208	-4.13%
11	104.448 / 104.448	102.400 / 102.400	-1.96%
12	112.640 / 112.640	112.640 / 112.640	+0.00%

Codec q4-q6, which that diff could not reach at all, was the second control: +0.00% at nine of the
eleven widths, +0.07% at T=3 (a single 0.032 us tick), and +5.18% at T=2. That last cell bounds
the instrument rather than the change: at T=2 the q4-q6 master arm disagreed with itself by 1.664 us
between its two passes, and the conservative delta rule turned that into a five-percent "regression"
on a codec the diff cannot touch.

Corrected. An earlier revision of that paragraph, and of the commit message, read: "Codec
q4-q6, which this change cannot reach at all, is the second control and reads exactly +0.00% at
all eleven widths." The attached data/moeop.txt shows +5.18% at T=2; its own footer repeats the
"exactly zero at all eleven widths" wording two lines below the table that contradicts it. Ten of
eleven is what was measured.

RETRACTED, 2026-09-05. A further paragraph read: "On master that spread is zero at every width
of the q4-q5 sweep except T=6 ... and zero at every width of the q4-q6 sweep except T=2. Those
are the only two non-reproducing cells in the whole table." Recounting all 22 cells of
data/moeop.txt: master fails to reproduce itself in five, not two. The three that were missed
differ by exactly one 0.032 us tick and were read as zero. The conclusion - that the two large cells
are the artefacts - survives; the premise was wider than the file.

Graph step and end to end, on e3aeaf8c and a140e7ae, at the threshold eleven: the round
dropped 2.05-2.24% at --spec mtp --draft-tokens 5 across contexts 128 / 4095 / 12000, and decode
rose +1.48% to +1.58% at that mode across contexts 128 / 1000 / 4095 / 12000, with drafts 0, 1 and 3
measuring -0.03% to +0.03%. Prefill was unaffected, -0.09% to +0.14%. The 2026-09-07 ladder above,
taken at 19 on a16b6442, reads +1.37% and +1.38% at the same mode - the same effect, on a different
base, at a different constant, measured with a different ladder shape. Neither number is a re-label of
the other.

Resource census, on ad0f3d38: both schedules the ladder can select already exist in
sparse_moe_decode_kernels.cu.o on master - sparse_moe_d4_token_kernel<Q5Codec, 2, false> at 43
registers / 72 B shared / 0 stack / 0 local, and <Q5Codec, 4, false> at 40 / 144 / 0 / 0. A
cuobjdump --dump-resource-usage census of that object, matched by demangled name with template
arguments, reported 0 changed bodies of 33. That census was not repeated on a16b6442. The
companion denominator for sparse_moe_small_t_kernels.cu.o was 95 on ad0f3d38 and is 50 on
this base, because PR #191 replaced the sparse_moe_small_t_s2_kernel<Tokens> family with one
non-template kernel - read out of the source, not out of a re-run.

Corrections published on the previous revision, kept here

One audit table and five correction blocks below stood in the published body of this PR. The
text each of them corrected has since been rewritten or re-measured on a16b6442, which is why they
no longer sit next to the sentences they belong to - but nothing this package has retracted is
allowed to disappear with the paragraph that carried it. They are reproduced verbatim, internal
words like "below" included, with the base each belongs to; the notes in italics between them are
new and say where each one now stands.

Line references re-checked, first against 487f8977 and again against a16b6442. The audit
below was published on the previous revision. a16b6442 is 487f8977 plus one documentation
commit, and I re-opened all five citations on a16b6442: every row still reads exactly as printed.

citation	printed before	correct number	what stands at the old number
src/targets/qwen3_6/impl/state/round_state.cpp, the DFlash allocation gate	:176	:174	:176 is inside the block, not its condition: DFlashDecodeStateLayout& decode = layout.dflash_decode.emplace();
include/ninfer/types.h, the backend default	:78	:79	:78 is the opening line struct SpeculativeOptions {
include/ninfer/types.h, ProposalHead::Full	:80	:82	:80 is the comment // Startup-fixed K: MTP 1..5; DFlash and DFlash2 1..15 (query width K+1).
src/product/speculative_options.h, the MTP draft-window cap	:38	:41	:38 is a closing }; the check `if (options.draft_tokens == 0
src/serve/serve_options.h, SpeculativeOptions speculative;	:46	:46	unchanged - re-checked, still correct

The gate that report quoted no longer exists, and the real one is wider. The published text
quoted if (layout.spec.enable_dflash). The field enable_dflash is gone from the tree entirely -
on a16b6442 git grep enable_dflash finds it only inside the frozen code samples under
eval/corpora/. round_state.cpp:174 now reads
if (is_masked_draft_backend(layout.spec.backend)), and that predicate is
backend == SpeculativeBackend::DFlash || backend == SpeculativeBackend::DFlash2
(src/targets/qwen3_6/export/ninfer/targets/qwen3_6/startup_features.h:7-9). So the gate admits
both --spec dflash and --spec dflash2, not --spec dflash alone, and the quotation and the
"reachable only through --spec dflash" wording have been corrected wherever they appeared. The
conclusion the paragraph draws is unaffected: the default is still SpeculativeBackend::None, and
no measurement in this report passes either flag. This is the first correction referred to in
part (a) of the Corrected block under Which widths the product can actually ask for.

Corrected. That divergence is over. On 487f8977 --mtp-draft-tokens does not exist:
ninfer_bench takes the same pair the CLI does, --spec <mtp|dflash|dflash2> together
with --draft-tokens N
(bench/targets/qwen3_6_27b/ninfer_bench_support.cpp:356-359,
apps/cli/options.cpp:141-145). Every command below has been rewritten to the current
spelling, and the zero-draft arm, which the old flag spelled --mtp-draft-tokens 0, is
now spelled by passing neither flag - --spec mtp --draft-tokens 0 is rejected.

(The 2026-09-07 commands in this revision all use the current spelling; the block above is why.)

Corrected. The Q5 interval read "(Q5 48.8-50.8%, Q6 50.8-53.4%)". 48.8% is the
T = 17 cell of the Q5 Rows2 column in the forced-arm crossover sweep - one width past the
Q5 crossover, i.e. exactly where Rows2 no longer wins, so it does not belong to an interval
qualified by "wherever Rows2 wins". Over the widths where Rows2 does win in that sweep -
T = 10..16 on Q5 and T = 10..17 on Q6 - the Rows2 column reads 50.1-50.8% on Q5
(50.4 / 50.5 / 50.6 / 50.7 / 50.8 / 50.8 / 50.1) and 50.8-53.4% on Q6, and the paired
Rows4 column reads 49.3-50.1% and 49.5-51.8% over the same widths. The Q6 interval was
already right; only the Q5 one was widened downward by a cell taken from the wrong side of the
crossover. The conclusion - both arms near half the read ceiling in the band, a third of it by
T = 46, so the kernel is latency-limited - is unchanged.

(Those percentages were taken on a140e7ae against the 1689.4 GB/s denominator this revision no
longer uses; see the roofline correction above. The 2026-09-07 roofline table replaces them and is
quoted against 1792.1 GB/s.)

Corrected: the denominator 95 no longer exists. 95 is the body count of
sparse_moe_small_t_kernels.cu.o at a140e7ae and ad0f3d38. The head of the current master,
487f8977 - our own merged PR #191 - rewrote small-T S2: the templates
sparse_moe_small_t_s2_kernel<Tokens> and sparse_moe_small_t_s2_two_batch_kernel<Tokens> are
gone, replaced by one non-template kernel. dispatch_tokens instantiates its lambda over
Tokens = 1..46, so on the old base that family emitted 46 bodies - s2_kernel<1..44> plus
s2_two_batch_kernel<45,46>, the split made by if constexpr (Tokens <= 44) - against 1 on
the new base. The census denominator therefore falls by 45, from 95 to 50. It checks out
against the old total: s1_kernel<1..46> is another 46 bodies, so the two families account for
92 of the 95 and the remaining 3 come from elsewhere in the translation unit and are untouched.
This is read out of the source, not out of a re-run.

Corrected. An earlier revision called T=6 "the smallest operator delta measured in the band
on this base, -2.99%". The smallest in the band is T=11 at -1.96%; the sentence read as a
conservatism claim that the table does not support.

What the wider window costs

Rows2 halves the destination rows one CTA serves, so the D4 grid doubles from kHidden/4 to
kHidden/2 CTAs per token (const dim3 grid = Adaptive ? dim3(kAdaptiveD4Blocks) : dim3(kHidden / Rows, tokens);, sparse_moe_decode_kernels.cu:619, kHidden = 2048 at :24). The FP32 activation
slab each CTA reads - (kTopK+1) x kIntermediate floats, 9 x 512 x 4 B = 18 KiB, from kTopK = 8 at
:27 and kIntermediate = 512 at :28 - is read outside the row loop, so halving Rows doubles how
many times that slab is fetched. It is L2-resident per token, so it does not appear on the DRAM
roofline, but it is the term that eventually outweighs whatever the narrower schedule wins, and it is
why a threshold exists at all. The measurement says that term does not win until T=20 on Q5 and T=21
on Q6.

Limitations and checks not run

• The mechanism is measured; the explanation is not. The stack this change came out of asserted
  that Rows4 costs 115 registers against 77 for Rows2 and therefore halves resident CTAs per SM.
  That does not reproduce on this tree. The census reads 40 registers for <Q5Codec, 4, false>
  and 43 for <Q5Codec, 2, false>, both with zero stack and zero local, and 144 B against 72 B of
  shared memory at 288 threads per CTA - i.e. the wider schedule uses fewer registers, and neither
  instantiation is near a register-driven occupancy cliff. The occupancy story is withdrawn. No ncu
  occupancy or stall-reason capture was taken in either session, so the proximate cause of the gain
  and the proximate cause of the crossover both remain unestablished.

• Why the Q5 crossover moved from 16 to 19 is not attributed. Between a140e7ae and a16b6442
  master rewrote sparse_moe_small_t_kernels.cu (102 lines) and sparse_moe_route.cuh (70 lines). I
  did not bisect. The shift is measured; its cause is not.

  Superseded. The previous revision's limitation bullet read: "Eleven is safe, but it is not
  the optimum ... The measured crossover is T=16 on Q5 and T=17 on Q6 ... I am not proposing the
  measured optimum in this PR, and the reason is procedural. The single constant that is exactly
  right for Q6 and free for Q5 is 17. Adopting it would invalidate every verification layer in this
  PR: the 18-of-18 bitwise gate, ctest, the graph step and both end-to-end ladders were all run at
  eleven, and I will not relabel numbers I did not take at the value I am proposing. It would also
  need end-to-end coverage of T=12..16, which is reachable only through --spec dflash or
  --spec dflash2 - backends this PR does not measure end to end." (The tail of that sentence is
  itself one of the claims corrected above: those widths are reachable from prefill too, and the
  "reachable only through --spec dflash" half was already withdrawn on the published revision.)
  Both halves of that are now settled rather than deferred:
  the crossover was re-measured on the current head (19 and 17, not 16 and 17), the verification
  layer was re-taken at 19 rather than relabelled, and both ends of T=12..16 are covered end to
  end through --spec dflash --draft-tokens 11 and 15 - T=13, 14 and 15 are not. The old bullet
  is kept because it is what this PR claimed.

• The full ctest was not run. 6 targets on both arms, out of 114 registered. That count was
  taken on this base, not carried: ctest.sh:6 runs
  cmake --build . --target help | grep _test$ and ctest -N | grep -c 'Test *#' in this tree's
  build directory, and logs/ctest_out.txt:1 records the result - 110 build targets, 114 ctest
  tests. (tests/CMakeLists.txt is byte-identical between 487f8977 and a16b6442 as well, so the
  two agree.) An earlier draft of this bullet claimed the count "was not re-counted by a ctest -N
  on this base", which the attached log contradicts. The build of the test set filled the machine's
  disk; see above. The remaining 108 were run on neither
  arm at this constant, and the claim that the registered set is unchanged is read out of the diff,
  not measured. No round "114 of 114" is claimed here for any base, for two reasons that predate this
  change: ninfer_attn_input_proj_test is red on the bare base (upstream defect, issue ninfer_attn_input_proj_test is flaky on master: the harness launches before the phase-1 activation upload has landed #196), and
  ctest and a direct run of the same test binary have been observed to disagree on this host.

• The prefill entry into this ladder has no end-to-end coverage at a width where the change acts.
  Prefill reaches this ladder with tokens = len for any chunk in [2,46] (see Which widths the
  product can actually ask for), so the constant is live for prompts of 19 tokens or fewer and for
  tail chunks of that size. The bitwise gate does exercise that entry - at 25 and 27 prompt tokens,
  which are above the constant, where both arms pick Rows4. Neither end-to-end ladder touches
  it at all: -pg 128,128 gives one chunk of 128 and -pg 4096,128 gives four chunks of 1024, and
  every one of those is >= 47, i.e. the prefill route rather than small-T. So the row
  "Prefill is -0.17 to +0.19%" in the ladder is measured at widths where this change cannot act;
  it is an internal control for the prefill route, not evidence about a short prompt. The live prefill
  widths are covered by the operator bench, which sets tokens directly, and by nothing else here.

• The roofline denominator is the vendor peak, not a measured ceiling. theoretical_memory = 1792.1 GB/s is what the benchmark prints and what the percentages above are taken against. A
  measured read ceiling for this card exists in our earlier notes as 1689.4 GB/s, but the probe behind
  it was not preserved and was not re-taken in this campaign, so no percentage here is quoted against
  it. Reading the same GB/s against a measured ceiling would raise every cell by about three points;
  the shape - both arms together, both sliding towards T=21 - does not depend on the choice.

• Batch > 1 has no end-to-end coverage. T = width x batch is established from the source, but
  neither the gate nor the ladder ran at batch > 1: the CLI and ninfer_bench both drive a single
  sequence. So the widths T=18..45 that only a batch can open are covered by the operator bench and by
  reading the code, not by an end-to-end run.

• --spec dflash2 was never exercised on this artifact. Its dflash section exists in the
  manifest (51 objects), but the dflash2 route (coherent_selector) was not run once.

• The gate does not see numeric corruption. The strength control proves it directly: 1 cell of 6
  passed a 0.1% corruption as IDENTICAL. No numeric oracle and no delta-NLL was taken here.

• The master2 control is not independent - it is a byte-identical rebuild, sound as a pipeline
  null control and nothing more.

• The top of the Q5 tail (T=26..30) is contradictory - +0.92% at the single point with overlapping
  samples, +-0.8..0.9% across windows without separation, which is the signature of a false signal.
  The first firm Rows4 win on Q5 is taken as T=30, not 26 and not 28. T >= 31 has no independent
  repeat: that stretch was swept by block G only. None of this touches the threshold, which sits at
  19.

• One artifact, one fixture. --distribution trace-like, default seed, Qwen3.6-35B-A3B. The D4
  grid does not depend on how routing concentrates, but D4's share of the operator does. Qwen3.6-27B
  has no routed MoE and never reaches this ladder, so it is untested rather than unaffected.

• The graph-step ladder and the cuobjdump resource census were not repeated on a16b6442.
  Both are carried from the old base and labelled where they appear.

• No test covers the ladder. SparseMoeSmallTD4Schedule appears in five sources
  (sparse_moe_decode.h, sparse_moe_decode_kernels.cu, sparse_moe_prefill_kernels.cu,
  sparse_moe_small_t.h, sparse_moe_small_t_plan.cpp) and two evaluation corpora, and in no file
  under tests/ - re-checked on a16b6442. A table-driven unit test over
  resolve_sparse_moe_small_t_plan across widths is about twenty lines and would pin the ladder to a
  contract. I have left it out to keep the diff to one hunk, and will add it in this PR if you want
  it; with the branch gone it is a shorter test than it was.

• The candidate was never built as a commit on the bench machine. The constant existed there only
  as a working-tree modification; the branch published here is the same one-hunk change applied to
  a16b6442.

Speculation counters: how much of this is round count and how much is round speed

ninfer_bench reports decode as generated tokens over elapsed time. A change that alters the numbers
the model produces can alter draft acceptance, and with it the number of speculative rounds spent
on a fixed output length - which raises tokens per second with no kernel running faster. The two
effects separate exactly, because the bench prints the round count:

decode_time(base) / decode_time(arm)  =  [ rounds(base) / rounds(arm) ]  *  [ t(base) / t(arm) ]

where t = decode_seconds_mean / spec_rounds. spec_rounds is an exact integer printed by the
engine and decode_seconds_mean is the measured time, so the split is an identity, not a model.

Here the round count is identical on both arms, so the whole figure is round speed. In the
2026-09-07 ladder, spec_acceptance_rate agrees between master and candidate to the third decimal in
all six speculative cells, and this holds at --spec dflash --draft-tokens 15, i.e. at T=16, the
widest width the change touches end to end - precisely where a round-count substitution would hide.
The earlier session's ladder made the same check across 16 groups of (draft window x point) at drafts
0, 1, 3 and 5, with spec_rounds, spec_fallback_steps and spec_acceptance_rate matching to the
last digit.

Which proposal head this was taken at. Every cell above ran at the product default
ProposalHead::Full (include/ninfer/types.h:82). --lm-head-draft, the configuration
docs/performance.md publishes in, is not covered by any cell in this package: an earlier revision
said it "was checked separately on the same artifact and gives the same split", and no file in
raw/ supports that. Treat every number here as ProposalHead::Full only.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-08T08:24:54.801397Z	01401e6	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 26a5f10c64

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Extend Q5 Rows2 through the measured winning interval

For Q5 routed-down at T=12..16, which is reachable through DFlash/DFlash2, this condition still selects Rows4 even though the change's forced-schedule sweep reports Rows2 is 1.47–1.82% faster there. Stopping at 11 solely because verification was performed at that boundary knowingly leaves an inferior production schedule; extend and requalify the boundary through the measured winning interval.

AGENTS.md reference: AGENTS.md:L7-L11

Useful? React with 👍 / 👎.

[MichaelDementii]

The Codex finding is correct, and the answer changed the change: 4f016015 replaces the branch.

Stopping at eleven was defended here as a verification-risk trade-off. AGENTS.md:7-11 says not to treat that as a reason to keep a known inferior design, so the sweep was re-taken on a16b6442 with a forced schedule instead of the threshold being argued about.

The crossover had moved. On the old base (a140e7ae) the measured optimum was 16 for Q5G64_F16S; on this base it is 19. Cells T=17, 18 and 19 read exactly +0.00% on the old base — the sweep honestly said there was nothing past 16 — and on this one they read -1.37%, -1.30% and -1.23% with full sample separation across five windows and at the single point. Q6 held at 17. The old measurement was not coarser; it was right for its base, which is why it raised no suspicion. Carrying the recorded answer forward would have left a measurable percent on a reachable width.

So the diff is no longer "raise Q5 to eleven". The routed_down branch is deleted and one constant serves both codecs: admissible intervals [19,25] and [17,20] intersect at [19,20], and 19 is Q5's measured optimum inside that. It also settles something the review did not raise — after the previous version of this PR, the two branches of the if were byte-identical, so the condition selected nothing.

Two claims of mine were wrong and are corrected in the body rather than quietly dropped.

The first was about which widths the product can reach. I had counted only the decode backends and concluded that 17, 19, 23 and other primes above 16 were unreachable, which made "18 versus 19 is indistinguishable" part of the argument. That is false: the small-T route is selected by token count alone, and the tail chunk of a prompt enters it as well (sparse_moe.cpp:208-210, sparse_moe_prefill_plan.cpp:24-28, text_context_impl.h:1107). The whole of [2,46] is reachable. The argument for 19 is simpler without it — measured optimum, reachable width — but the consequence is not in my favour and is now stated: the change is live in prefill of short prompts, and neither end-to-end ladder here exercises that entry.

The second was the null control. The body said 40 of 40 cells read exactly +0.00%. Three read +0.02%, and two of those three sit at T=19, the width this PR proposes as the constant. Worse, ±0.02% is the floor of the median delta, not of the cell: the block-Z cell at T=19 carries 1.23% and 1.22% spreads, so read worst-pass against best-pass the same null cell is +1.22% — a whole lattice step, at the constant. It fails the separation half of the rule and is read as zero; the rule used throughout is now stated as full separation plus a whole 2.048 us step, not magnitude alone.

What is bought, and what it costs if the contested reading is wrong. Q5 T=12..19 gains 1.23% to 1.82% of the operator, on 37 of the 40 MoE layers. End to end at the constant: decode +1.38% at T=6, +0.89/+0.67% at T=12, +0.44/+0.43% at T=16; prefill zero; the no-speculation row zero. The Q6 contest at T=18,19 was resolved in this change's favour by taking three-of-five non-zero windows as lattice noise. If that is wrong, Q6's interval shrinks to [17,17], does not intersect Q5's at all, and 19 becomes an explicit trade — about 1.2% given up on three Q6 layers against 1.23-1.30% gained on thirty-seven Q5 layers. Inside that contest one cell argues the other way and is printed rather than summarised: in window K at T=19 the Rows2 arm is stationary at 172.032 us while Rows4 moves, four of its five passes a whole step below it.

Verification was re-taken at 19, which is what the previous version could not claim: bitwise gate 24 of 24 IDENTICAL with a strength control at 5 of 6 DIFFER, covering T=12..16 through --spec dflash --draft-tokens 11/15; the end-to-end ladder and round step above; clang-format clean.

Not done. The full ctest was not run — building the 110 test targets filled the stand's disk; a targeted subset of six passed on both arms, and the registered set is unchanged by inspection, not by measurement. Why the crossover moved between bases was not bisected, and no counter capture was taken, so no mechanism is claimed — the code comment states the measured thresholds and says explicitly that the cause was not measured. Batch above one end to end, --spec dflash2 on this artifact, and the prefill entry are untested rather than unaffected.

🤖 Generated with Claude Code

[MichaelDementii]

The branch was replaced after the finding above; the review on this PR still points at the previous head. Current head is 4f016015.

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4f016015bd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve the measured Q6 crossover

For the Q6 profile at T=18 or T=19, this selects Rows2 even though the replacement commit's own forced-schedule data puts the Q6 crossover at 17 and reports roughly 1.2% wins for Rows4 in three of five windows at those widths, with full sample separation in three of the six readings. These widths are reachable by batched/speculative decode or small-T prefill, and the parent selected Rows4, so collapsing the existing codec branch introduces a measured Q6 regression; retain the independently measured Q6 threshold while extending Q5 to 19.

AGENTS.md reference: AGENTS.md:L7-L11

Useful? React with 👍 / 👎.

[MichaelDementii]

Correct, and taken: 01401e6c. Q5 goes to 19, Q6 to 17, and the routed_down branch stays.

The objection lands on the one joint in the previous revision that rested on a judgement rather than on a number. Collapsing the branch needed the intervals [19,25] and [17,20] to intersect, and that intersection exists only under the reading that takes Q6's contested cells at T=18 and T=19 as lattice noise. The report says as much in its own words — three of five windows at each width read about +1.2% for Rows4, three of those six readings with full sample separation — and priced the consequence at about 1.2% on three Q6 layers if the reading is wrong.

What makes it clear-cut is that the bet had nothing on the other side. 17 is not worse than 19 for Q6 under either reading: if the contested cells are noise, T=18 and T=19 run identically either way; if they are signal, 17 is right and 19 is a regression. There is no reading in which the shared constant wins on Q6, so it bought only tidiness, and AGENTS.md:7-11 is explicit that diff size is not a reason to keep a design known to be inferior. The Q5 answer is untouched at 19.

One thing from the previous revision is kept, because the measurement supports it: the branch as it stood on master was not supported either. It separated the codecs by six tokens (5 against 11) where this base's sweep separates them by two (19 against 17). Both arms move up; what changes now is that each moves to its own measured value instead of to a shared one.

What was re-run and what was not. The diff is one file, +10/−9, clang-format clean, and it applies to a16b6442 on a fresh worktree. The bitwise gate, the end-to-end ladder and the round step were taken at 19/19 and have not been re-taken at 19/17. Two of the three transfer by inspection and the third does not:

• The end-to-end ladder measures T=6, 12 and 16, all below both thresholds, so the Q6 arm runs Rows2 under either constant and those numbers are unaffected.
• The bitwise gate compared master against the candidate and read 24 of 24 IDENTICAL. Lowering the Q6 threshold to 17 makes the candidate agree with master at T=18 and T=19 where it previously differed in schedule, so the comparison it passed is strictly easier, not harder.
• The Q6 rows of the forced-schedule sweep are the measurement this revision now follows rather than overrides, so nothing there needs re-taking — but no fresh end-to-end run was made at 19/17, and I am not claiming one.

🤖 Generated with Claude Code

[MichaelDementii]

Head is now 01401e6c (Q6 kept at its own measured 17); the review above ran on 4f01601.

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 01401e6c90

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Neroued]

Local review of 01401e6c90b552a91da35806a1d68b62cb8cae12 against baseline 437e9f980a763caf73d2e296a60b1abd94b97436: I do not recommend merging this version because the Q6 Rows2 extension through T=17 regresses a supported routing case.

On RTX 5090, CUDA 13.1, Release sm_120a (g++ 13.3, Ubuntu 24.04/WSL2), --codec q4-q6 --distribution same --tokens 17 (the same eight experts across tokens) gave:

Arm	Median public SparseMoe Op time
Baseline, Rows4	127.008 us
Candidate, Rows2	129.056 us
Identical baseline control	127.008 us

This is a 1.6125% Op-time regression, with zero difference between the identical baseline controls. The controlled experiment captured all three graphs in one process over shared inputs/buffers and interleaved replay through all six arm orders: 300 samples per arm after 20 warmups, with a 256 MiB cold-cache flush before each call. Three separate-binary comparisons also showed +1.613%, +1.613%, and +1.587%. At trace-like T=17, the same-process comparison showed no gain either (153.376 vs 153.600 us).

The rest of the change has useful results: Q5 trace-like T=6..19 reduced Op time by approximately 1.3–5.9%, and Q6 T=12..16 improved approximately 1.2–1.8%. The original 16 numerical cases and 58 supplemental FP64 cases passed on both arms; the complete BF16 outputs of the supplemental cases were byte-identical. No correctness or implementation-quality blocker was found.

Please requalify the Q6 upper boundary, including a cutoff of 16, or use a justified selection that avoids the T=17 regression. This does not establish that 16 is globally optimal; the Q5 improvement can be retained independently. A focused comparison at the boundary with both concentrated and trace-like routing should be sufficient to revisit this finding.

This is an Op-level performance finding, not a mathematical bug or a claim of 1.6% whole-model slowdown. End-to-end measurements were not pursued after establishing this changed-route regression. These measurements belong to the baseline above, not today's master.

[MichaelDementii]

You were right. Reproduced first, then requalified; the change is now smaller than the one you suggested.

(Reposted: the first version of this comment lost its non-ASCII characters in transit.)

Your cell, reproduced

--codec q4-q6 --tokens 17 --distribution same, four passes per machine, arms mirrored inside a pass:

	Rows4	Rows2	delta	identical-binary control
you	127.008	129.056	+1.61 %	0
rented 5090, quiet Linux	131.072	133.120	+1.56 %	0.00 %
local, WSL2	174.112	178.176	+2.33 %	0.0000 %

Your question about a cutoff of 16

Tested. 16 is right on one of my two machines and not on the other, so I took the justified
alternative you allowed instead. Q6, concentrated routing, median of the kept per-pass ratios:

T	local, 1975 MHz	rented, 2810 MHz
11	+0.03 %	-2.22 %
12	+1.56 %	+0.00 %
13	+0.02 %	-0.02 %
14	+1.37 %	-0.94 %
15	+1.30 %	+0.00 %
16	+2.45 %	+0.00 %
17	+1.33 %	+1.56 %

On the rented box 16 is exactly the boundary, matching your finding. On the local card the losses
start at T=12, so a cutoff of 16 would still regress T=12, 14, 15 and 16 there. The value that
holds on both is 11, which is what master already has, so Q6 is left untouched.

The interleaved ties at T=13 on the local card are not noise: nine passes, agreeing to two
decimals, with the identical-binary control reading zero in the same cells. I have no mechanism
for the non-monotonicity and am not claiming one.

What the PR is now

One number: the Q5G64_F16S window goes 5 -> 17. Q6G64_F16S stays at 11. The 19 in v1 was also
too high: on the lower-clocked machine Q5 breaks at 18.

This matters per codec because routed_down_format in Qwen3.6-35B-A3B picks Q6G64_F16S for layers
34, 38 and 39 and Q5G64_F16S for the other 37, so both branches run in the same decode round. v1
would have regressed three layers in order to widen the window for the rest.

Requalification

Three arms from one tree: Rows2 forced, Rows4 forced, and a second build of the Rows4 source that
came out byte-identical to the first and serves as the null control. Both codecs, all three
routing distributions, T = 2..22, up to nine passes, two machines. Arms compared paired inside a
pass; a pass whose two identical binaries disagree in a cell is discarded for that cell.

Largest T at which Rows2 is not slower (threshold 0.50 %, T <= 2 excluded, that is Rows1):

codec	trace-like	independent	same
Q5, local	22	18	17
Q5, rented	22	20	19
Q6, local	15	16	11
Q6, rented	17	22	16

same binds everywhere, as you found.

Why the two machines differ, and which one sets the value

The local card sits at 1975 MHz under this load at 48 C and 265 W of a 575 W cap - not
thermally or power limited, it simply does not boost; the rented one runs 2810 MHz at 476 W.
The cost of Rows2 is SM-side, so at a lower clock it is larger in absolute time and the crossover
moves left. The lower-clocked reading is therefore the binding one.

Mechanism

Rows is only a grid split, (kHidden / Rows, tokens). Counters on the D4 kernel, Q6 at T=17,
four launches per arm:

	Rows2 / Rows4
blocks	x2.000
l1tex__t_bytes.sum	x1.634
lts__t_bytes.sum	x1.004
sm__throughput	x0.841
duration	x1.142

Rows2 buys parallelism to cover DRAM latency while the grid is small, and pays by re-reading the
same per-expert activations in every block. The extra traffic is absorbed by L1; L2 is flat.
dram__* is not exposed on this part (returns n/a), so the DRAM statement rests on
lts__t_bytes not moving.

Two things v1 got wrong beyond the number

1. It swept one routing distribution and presented the result as general. The one you used is the
   one that binds.
2. It explained the contested T=18,19 cells as "2.048 us lattice noise". That was not noise: four
   independent processes give the same median to the bit and the null control reads exactly zero.
   Those cells were signal.

I also checked whether the base had moved the crossover - two commits touch this kernel between
ad0f3d38 and today's master, both adding L2 prefetch. It had not: the same comparison on
ad0f3d38 gives +1.51 % at T=12 and +2.23 % at T=17 against +1.56 % and +2.35 % here. Those
commits lowered absolute time by about 4 % without moving the balance between schedules. The old
boundary was simply wrong.

Verification and what is not offered

Bit-exact by construction: Rows only changes how output rows are distributed over blocks; each
row accumulates over the intermediate dimension in the same order, and the nine paths are summed
in rank order in the epilogue. Greedy token ids over 128 tokens are identical to master on the 35B
artifact. ctest is 114/114 on this branch and 114/114 on master, run from the same worktree
with the same flags (your b88c0f6f cleared the one failure that used to be there).

No end-to-end number. At batch 1 the decode round runs T=4, inside both the old and the new
window; the range this touches, T=6..17, needs batch 2 and above, and neither machine here resolves
a per-layer operator effect of this size end to end.