Repository navigation
blackwell-16: serve the 26B with --n-cpu-moe 14 (measured on a display-card RTX 5080); retire the CUDA-13 cuBLAS-fallback warning - #751
Merged
Conversation
…display-card RTX 5080; retire the CUDA-13 cuBLAS-fallback warning Full GPU needs about 14.4 GB on the card and spills whenever the desktop holds more than about 1.5 GB (17 tok/s against 139 once at an empty card). N=14 keeps 9.1 GB on the card at 62.5 tok/s and fits under the heaviest desktop observed (5.4 GB). The same run's llama-bench on the win-cuda-13.4 prebuilt b12110 (E4B pp512 17694 tok/s) retires the 2026-07-15 cuBLAS-fallback finding, so detect.ps1 no longer warns about it and the installer's build report says what was measured. Report, SETUP-AGENT note and tier pages updated. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… 26B; tests follow the measured placement The RAM gate drops an n_cpu_moe 26B on a box under 28 GB, so config_seed.agent_model is an explicit blank (the lane follows the workhorse) and config_seed_ram_low_up names gemma-4-26b-agent on 28 GB and up; tierseed.Resolve applies the same rule to a vLLM seat's llama.cpp fallback (ramDrops26B mirrors the render gate). The spill-seat test allows a tier's own measured partial spill, the closure lint evaluates the bands the gate drops the 26B on, the gate and audit tests start from the placement they name, the installer self-test expects --n-cpu-moe 14 on both 26B entries, and the build-tag text the no-pins lint flagged is gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… RAM band The base seed blanks agent_model explicitly and config_seed_ram_low_up seats gemma-4-26b-agent on 28 GB and up, so the 16 GB-tier loop asserts exactly that for blackwell-16. Suite run locally under Windows PowerShell 5.1: ALL PASS (760 checks). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
blackwell-16ships a 26B placement that fits a 16 GB card that is also the desktop's display card, decided by measurement on the tier's new reference box (docs/tiers/reports/blackwell-16-node-h.md), and the Blackwell build texts say what that measurement found instead of a finding from an earlier build.Scope
setup/templates/profiles.json:blackwell-16goes frommoe_26b: gputon_cpu_moewith N=14 andn_cpu_moe_max14; its notes carry the measurements and the reference box; then_cpu_moe_maxdoc line names the second tier that declares one. Because the RAM gate drops ann_cpu_moe26B on a box under 28 GB, the agent seat binds by RAM band:config_seed.agent_modelis an explicit blank (the lane follows the workhorse on a min box) andconfig_seed_ram_low_upnamesgemma-4-26b-agenton 28 GB and up.internal/tierseed/tierseed.go:Resolveno longer binds a vLLM seat's llama.cpp fallback when it names the 26B agent seat on a RAM band whose gate drops the 26B (ramDrops26BmirrorsmoePlacement/Resolve-ProfileParams); the seed layers decide the lane there. Onlyblackwell-16declares a 26B fallback.setup/detect.ps1: the CUDA-13 branch no longer adds the "cuBLAS fallback, ~5.6x slower prefill" warning; the 12.8 and undetected texts match.setup/install.ps1: the build matrix comment and theSelect-CudaBuildreport lines carry the measured numbers.setup/SETUP-AGENT.md: the Blackwell note, same correction.docs/tiers/reports/blackwell-16-node-h.md: an "Operator measurements" section after the report proper (bench, the placement sweep, the live result).docs/tiers/README.md,docs/tiers/blackwell-16.md: regenerated (go generate .).TestNoOtherTierRendersTheSpillSeatOrAnNCPUMoESpill(a tier's own measured partial spill may carry--n-cpu-moe; the spill seat stays ampere-6's),TestNo26BAgentSeatOnRAMDroppable26B(evaluates the RAM bands the gate drops the 26B on, with the seed layers that apply there),TestInstallRenderRefusesNCPUMoEAboveTheTiersMeasuredSpill(clears the table's own fields per case), the two audit tests (the bare--cpu-moecontrol edit anchors on the new 26B line),setup/render.tests.ps1(expects--n-cpu-moe 14 -ngl 999on both 26B entries).Left out on purpose: the 8 GB Blackwell tier's note still says it requires the 12.8 build (not measured here); no new seat for Qwen3.8-27B on Windows (measured as an option only); no template change (both 26B entries already take the placement token).
What changed
Three commits: f7589b0 (the placement, the texts, the report), c038049 (answers the first CI run: the band-aware agent seat,
tierseed.Resolve, the tests above, the build-tag text the no-pins lint flagged) and 2da7bbb (the installer's config-seed self-test reads the agent seat by RAM band;ALL PASS, 760 checks, locally under Windows PowerShell 5.1). The second CI run's other red check wasinternal/mediaremote'sTestAnUnknownNodeIsAContractRefusalListingTheRoster(a loopback test node did not answer its health GET on the runner); that package is untouched here and passed on the first run.The real run and its output
The measurement (same box, llama.cpp win-cuda-13.4 prebuilt, desktop at 4.5 GB falling to 1.6 GB during the run):
-ngl 99(as rendered before)--n-cpu-moe 4--n-cpu-moe 8--n-cpu-moe 14llama-bench on the same build: E4B pp512 17694 tok/s / tg128 212, E2B 34454 / 345. No cuBLAS fallback.
The renderer, run on this branch against main as the control (
go run . install render -profile blackwell-16 -os windows -ram-tier high ...):The placement live on the box (both entries patched to the rendered form, llama-swap restarted while idle), one chat completion on
gemma4-26b-a4bat the owner's desktop:After the follow-up commit, locally:
go test ./internal/tierseed/...ok; the root package's seed, tier, agent, closure, audit and no-pins tests ok (-run 'FleetCap|Seed|Tier|Agent|...', 17 s);setup/render.tests.ps1under Windows PowerShell 5.1 with the branch renderer:ALL PASS(326 checks). Gates once:go build ./...,go vet, the profile parses,go generate .clean.Blast radius
blackwell-16render: the 26B cascade and agent entries change their placement; about 5 GB of experts move to host RAM (the tier's RAM gate still drops the 26B below 28 GB of RAM, as for anyn_cpu_moetier, and such a box now runs the agent lane on the workhorse instead of a dropped seat). A headless 16 GB card loses the full-GPU speed it could reach on an empty card (noted in the profile; N=4 measured there).tierseed.Resolve: only a vLLM fallback naming a 26B seat on a band that drops the 26B changes, which isblackwell-16on min RAM and nothing else in the table.detect.ps1on a Blackwell box with CUDA 13.x: one warning fewer.Risk
Low. jev review-needed =
no_review(consequential 0.12, blast 1.75): standard tier, no review pass.🤖 Generated with Claude Code