Skip to content

blackwell-16: serve the 26B with --n-cpu-moe 14 (measured on a display-card RTX 5080); retire the CUDA-13 cuBLAS-fallback warning - #751

Merged
dmmdea merged 3 commits into
mainfrom
tier/blackwell-16-measured
Nov 4, 2026
Merged

dmmdea merged 3 commits into
mainfrom
tier/blackwell-16-measured

Conversation

@dmmdea

@dmmdea dmmdea commented Nov 4, 2026 •

Copy link
Copy Markdown
Owner

Intent

blackwell-16 ships a 26B placement that fits a 16 GB card that is also the desktop's display card, decided by measurement on the tier's new reference box (docs/tiers/reports/blackwell-16-node-h.md), and the Blackwell build texts say what that measurement found instead of a finding from an earlier build.

Scope

  • setup/templates/profiles.json: blackwell-16 goes from moe_26b: gpu to n_cpu_moe with N=14 and n_cpu_moe_max 14; its notes carry the measurements and the reference box; the n_cpu_moe_max doc line names the second tier that declares one. Because the RAM gate drops an n_cpu_moe 26B on a box under 28 GB, the agent seat binds by RAM band: config_seed.agent_model is an explicit blank (the lane follows the workhorse on a min box) and config_seed_ram_low_up names gemma-4-26b-agent on 28 GB and up.
  • internal/tierseed/tierseed.go: Resolve no longer binds a vLLM seat's llama.cpp fallback when it names the 26B agent seat on a RAM band whose gate drops the 26B (ramDrops26B mirrors moePlacement / Resolve-ProfileParams); the seed layers decide the lane there. Only blackwell-16 declares a 26B fallback.
  • setup/detect.ps1: the CUDA-13 branch no longer adds the "cuBLAS fallback, ~5.6x slower prefill" warning; the 12.8 and undetected texts match.
  • setup/install.ps1: the build matrix comment and the Select-CudaBuild report lines carry the measured numbers.
  • setup/SETUP-AGENT.md: the Blackwell note, same correction.
  • docs/tiers/reports/blackwell-16-node-h.md: an "Operator measurements" section after the report proper (bench, the placement sweep, the live result).
  • docs/tiers/README.md, docs/tiers/blackwell-16.md: regenerated (go generate .).
  • Tests that pinned the old placement, each moved to the rule rather than deleted: TestNoOtherTierRendersTheSpillSeatOrAnNCPUMoESpill (a tier's own measured partial spill may carry --n-cpu-moe; the spill seat stays ampere-6's), TestNo26BAgentSeatOnRAMDroppable26B (evaluates the RAM bands the gate drops the 26B on, with the seed layers that apply there), TestInstallRenderRefusesNCPUMoEAboveTheTiersMeasuredSpill (clears the table's own fields per case), the two audit tests (the bare --cpu-moe control edit anchors on the new 26B line), setup/render.tests.ps1 (expects --n-cpu-moe 14 -ngl 999 on both 26B entries).

Left out on purpose: the 8 GB Blackwell tier's note still says it requires the 12.8 build (not measured here); no new seat for Qwen3.8-27B on Windows (measured as an option only); no template change (both 26B entries already take the placement token).

What changed

Three commits: f7589b0 (the placement, the texts, the report), c038049 (answers the first CI run: the band-aware agent seat, tierseed.Resolve, the tests above, the build-tag text the no-pins lint flagged) and 2da7bbb (the installer's config-seed self-test reads the agent seat by RAM band; ALL PASS, 760 checks, locally under Windows PowerShell 5.1). The second CI run's other red check was internal/mediaremote's TestAnUnknownNodeIsAContractRefusalListingTheRoster (a loopback test node did not answer its health GET on the runner); that package is untouched here and passed on the first run.

The real run and its output

The measurement (same box, llama.cpp win-cuda-13.4 prebuilt, desktop at 4.5 GB falling to 1.6 GB during the run):

placement desktop before (MiB) seat on the card (MiB) free after load (MiB) pp tok/s tg tok/s
-ngl 99 (as rendered before) 4546 10997 + spill 760 351 17.0
--n-cpu-moe 4 1546 13065 1692 1428 121.2
--n-cpu-moe 8 1553 11563 3187 824 81.2
--n-cpu-moe 14 1556 9109 5638 535 62.5

llama-bench on the same build: E4B pp512 17694 tok/s / tg128 212, E2B 34454 / 345. No cuBLAS fallback.

The renderer, run on this branch against main as the control (go run . install render -profile blackwell-16 -os windows -ram-tier high ...):

branch: line 54  --n-cpu-moe 14 -ngl 999 --parallel 1 ${common}
        line 66  --n-cpu-moe 14 -ngl 999 --parallel 1 --ctx-size 32768 --flash-attn on
main:   0 occurrences of n-cpu-moe 14

The placement live on the box (both entries patched to the rendered form, llama-swap restarted while idle), one chat completion on gemma4-26b-a4b at the owner's desktop:

26B via llama-swap: wall=8.5 s (cold load included) pp=472.2 tok/s tg=58.2 tok/s prompt_n=796 gen=30
used with the 26B loaded: 10678 MiB, 16303 MiB

After the follow-up commit, locally: go test ./internal/tierseed/... ok; the root package's seed, tier, agent, closure, audit and no-pins tests ok (-run 'FleetCap|Seed|Tier|Agent|...', 17 s); setup/render.tests.ps1 under Windows PowerShell 5.1 with the branch renderer: ALL PASS (326 checks). Gates once: go build ./..., go vet, the profile parses, go generate . clean.

Blast radius

  • Every blackwell-16 render: the 26B cascade and agent entries change their placement; about 5 GB of experts move to host RAM (the tier's RAM gate still drops the 26B below 28 GB of RAM, as for any n_cpu_moe tier, and such a box now runs the agent lane on the workhorse instead of a dropped seat). A headless 16 GB card loses the full-GPU speed it could reach on an empty card (noted in the profile; N=4 measured there).
  • tierseed.Resolve: only a vLLM fallback naming a 26B seat on a band that drops the 26B changes, which is blackwell-16 on min RAM and nothing else in the table.
  • detect.ps1 on a Blackwell box with CUDA 13.x: one warning fewer.
  • Docs otherwise.

Risk

Low. jev review-needed = no_review (consequential 0.12, blast 1.75): standard tier, no review pass.

🤖 Generated with Claude Code

dmmdea and others added 3 commits November 4, 2026 15:39
…display-card RTX 5080; retire the CUDA-13 cuBLAS-fallback warning

Full GPU needs about 14.4 GB on the card and spills whenever the desktop holds more than about 1.5 GB (17 tok/s against 139 once at an empty card). N=14 keeps 9.1 GB on the card at 62.5 tok/s and fits under the heaviest desktop observed (5.4 GB). The same run's llama-bench on the win-cuda-13.4 prebuilt b12110 (E4B pp512 17694 tok/s) retires the 2026-07-15 cuBLAS-fallback finding, so detect.ps1 no longer warns about it and the installer's build report says what was measured. Report, SETUP-AGENT note and tier pages updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… 26B; tests follow the measured placement

The RAM gate drops an n_cpu_moe 26B on a box under 28 GB, so config_seed.agent_model is an explicit blank (the lane follows the workhorse) and config_seed_ram_low_up names gemma-4-26b-agent on 28 GB and up; tierseed.Resolve applies the same rule to a vLLM seat's llama.cpp fallback (ramDrops26B mirrors the render gate). The spill-seat test allows a tier's own measured partial spill, the closure lint evaluates the bands the gate drops the 26B on, the gate and audit tests start from the placement they name, the installer self-test expects --n-cpu-moe 14 on both 26B entries, and the build-tag text the no-pins lint flagged is gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… RAM band

The base seed blanks agent_model explicitly and config_seed_ram_low_up seats gemma-4-26b-agent on 28 GB and up, so the 16 GB-tier loop asserts exactly that for blackwell-16. Suite run locally under Windows PowerShell 5.1: ALL PASS (760 checks).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@dmmdea
dmmdea merged commit 5be6f68 into main Nov 4, 2026
6 checks passed
@dmmdea
dmmdea deleted the tier/blackwell-16-measured branch November 4, 2026 21:36
@dmmdea dmmdea mentioned this pull request Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant