docs(benchmarks): node1 GLM-5.2 datapoints, and a paired M4 Pro cap comparison - #1272
Open
lornecodes wants to merge 1 commit into
Open
docs(benchmarks): node1 GLM-5.2 datapoints, and a paired M4 Pro cap comparison#1272lornecodes wants to merge 1 commit into
lornecodes wants to merge 1 commit into
Conversation
…mparison Five rows for the 12900K + 990 Pro box (commit-stamped 72d3d37, July 2026): the MTP=0 result with its mechanism (600 experts/token against 1,062 with MTP on), the CUDA expert tier separated from the config change it rode in with, a two-drive file-level split, and six consecutive REPIN rounds showing where learned pins converge. One row for an M4 Pro running OLMoE fully resident, comparing cap 32 against cap 64 in interleaved pairs: cap 32 wins 7 of 8, median 19.1 against 16.3 tok/s, at half the resident footprint. Two notes. The cap comparison is paired because unpaired sweeps on that host do not agree with each other about the sign -- one five-run sweep put cap 64 ahead, a later ten-run block put cap 32 ahead by 35%, and cap 32 alone ranged 13.97 to 20.62 across sessions. Same-session alternation is the only thing that resolves it, which is the control the decode failure ledger already asks for. And the file-level split is recorded as the negative half of JustVugg#1249: same class of hardware, +5.5% against +37.5%, because splitting whole shards only parallelises reads that are already concurrent and that box sat at queue depth ~1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Datapoints from two machines, plus two notes where the negative result is the useful half.
node1 — i9-12900K + Samsung 990 Pro, GLM-5.2
Five rows, commit-stamped
72d3d37(July 2026) since they predate a lot of movement:MTP=0with its mechanism — 600 experts/token against 1,062 with MTP on, which is why speculation is a wash on a disk-bound tier: it saves 2.29x the forwards and pays 77% more expert loads for themREPIN=1rounds: 0.404 → 0.405 → 0.456 → 0.458 → 0.433 → 0.461, converging in ~3 rounds and holdingM4 Pro — OLMoE, fully resident, cap 32 vs 64, paired
cap 32 wins 7 of 8 interleaved pairs, median 19.1 vs 16.3 tok/s, at half the resident footprint.
The pairing is not a formality and the first draft of this was wrong without it. Unpaired sweeps on that host disagree about the sign: one five-run sweep put cap 64 ahead (14.96 vs 14.32), a later ten-run block put cap 32 ahead by 35% (18.21 vs 13.45), and cap 32 alone ranged 13.97–20.62 across sessions. Absolute throughput drifts more between sessions than the effect being measured, so only same-session alternation resolves it — the control the decode failure ledger already asks for after WarpDecode. Related to #1050 but the opposite shape; commented there.
The split-disk row is a negative, deliberately
+5.5% against #1249's +37.5%, on the same class of hardware — two NVMe on independent controllers. The gap is mechanism, not hardware: splitting whole shards across two mounts with symlinks only parallelises reads that are already concurrent, and that box sat at queue depth ~1 with the drive 24-28% utilised. Block striping and the engine's own weighted mirror parallelise within each 19 MB read. Recorded because it prices the difference between splitting files and striping blocks at roughly 5% versus 37%.
Raw logs for the node1 rows are kept and can be attached if wanted.