gpu: geglu_tanh, the GELU-gated unit over f32 rows

[juliankang4]

Part of #59 (plan and PR order in #190), E4: the f32 GELU-gated entry. The head-512 prefill entries and the prefill window are separate PRs.

What and why

Release notes: Added: an f32 GELU-gated kernel for EmbeddingGemma 2's feed-forward; no engine path calls it yet.

EmbeddingGemma's feed-forward is a GELU-gated linear unit: llama.cpp builds it with build_ffn(..., LLM_FFN_GELU, LLM_FFN_PAR), which calls ggml_geglu_split(gate, up), and its CUDA kernel computes gelu_tanh(gate) * up with op_gelu. The engine had the tanh GELU only as a bf16 in-place kernel inside gpu-vision (vis_gelu_tanh). This PR adds the f32 gated unit to bloomery-gpu as a kernel, geglu_tanh, so a text model's FFN can use it.

The PR has two commits:

1. Move. The f32 rule moves out of crates/gpu-vision/src/gelu_tanh.rs into a new crates/gpu/src/gelu.rs: the host function gelu_tanh_f32 and the device core gelu_tanh_dev (a core outside #[cuda_module], docs/gpu-design.md decision 6). vis_gelu_tanh and the MiniCPM-V host path now import them from bloomery_gpu::gelu. No kernel changes.
2. Add. elem.rs's module gains one entry beside swiglu, geglu_tanh(gate, up, n, y), and ElemKernels gains enqueue_geglu. The rule on the gate value is gelu_tanh_dev, then one rounded multiply by the up value, the same order as unary_gated_op_kernel at op_gelu. An empty launch or a short gate, up or y buffer is refused by name. The gate test lives in gelu.rs, beside the rule it holds the kernel to.

No engine path calls the kernel yet. The E5 body of #190 (gpu-gemma-embedding2) is its first caller; it reaches the entry through the ElemKernels the Gpu already loads.

Code shape

• No second copy. The f32 rule has one owner, bloomery_gpu::gelu. gpu-vision keeps only its bf16 in-place kernel, which calls the shared core.
• Model code holds only architecture facts. No model code changes.
• Dead code is gone. The vision file's own constants and host function are removed.
• No silent failure. A NaN gate or up value gives NaN at that index, also where a saturated negative gate would make the product 0 (the gate checks both), and enqueue_geglu refuses n == 0 and short buffers.
• Comments state what is true now.
• No relaxed gate. tools/unsafe-ratchet.txt moves gpu from main's 1787 to 1788 blocks with a PIN(2026-10-15): line: the new kernel's Elem2::new, with its SAFETY line.
• cargo fmt and cargo clippy are clean on what I touched; the lint count did not go up.

Duplication found or removed (path:line): crates/gpu-vision/src/gelu_tanh.rs (the f32 rule, its two constants and the host twin) now uses the copy in crates/gpu/src/gelu.rs. geglu_tanh and enqueue_geglu (crates/gpu/src/elem.rs:1117, :2177) follow swiglu and enqueue_swiglu (:1094, :2148) line for line except the activation, as the module's other elementwise entries do. They share the module, its load and its unsafe load block; a body generic over the activation would add more than the two short copies cost.

Remaining: the gate test's U and gamma(n) (crates/gpu/src/gelu.rs:41) repeat crates/gpu-gates/src/rounding.rs. bloomery-gpu cannot take bloomery-gpu-gates as a dev-dependency: bloomery-gpu-gates depends on bloomery-gpu, and with the reverse edge tools/recipes.py opens refuses every gate list (recipes.py opens: bloomery-gpu depend on bloomery-gpu-gates), which turns check-recipes red. One owner would mean moving rounding below bloomery-gpu, a change to every gate that imports it.

What I ran

The branch is now 6cdf0b69e2678b642a4c7460fe79c04354b7d923 on top of d0cbd4bbd109d92a7a0c5a0001029a91ce488a57. That rebase changed none of this PR's own lines: tools/unsafe-ratchet.txt takes gpu from main's 1787 blocks to 1788, and the third commit adds the gates main's new code reaches to the crates/gpu/src/gelu.rs row of tools/gate-paths.tsv. At this head I reran the gate and just mac-static (below). The other runs are at the head before the rebase, 618aec3f1da4d5525df9c56b537ab200dcf57194, on top of e98d6c43f95bc75475cadaf1043c1735962dcc5d, unless a line names another commit. In the Rust sources and manifests this PR changes, the head is the same as 30f18e63 (the branch before its rebase onto e98d6c43), differs from b689bc93 only in gelu.rs's test module (one short-buffer case there, three at the head), and from 0eb4a353 only in that module's U and gamma and the dev-dependency in crates/gpu/Cargo.toml and Cargo.lock. The card runs used an RTX 5090 (x86_64 Linux) on the host and a DGX Spark (aarch64 Linux) in the project's dev container. The static checks ran on an Apple Silicon Mac.

just contrib-check --tier t0 on the RTX 5090:

DONE tier=t0 total=82 red=1 wall=219s red: check-recipes

The one red item is a self-test of the recipe checks, FAIL stall-term: got [post 0 rc=124], want [post 0 rc=143]. The base e98d6c43 fails the same self-test, and nothing else, on the same machine (DONE tier=t0 total=82 red=1 wall=218s red: check-recipes).

just mac-static on the Mac at 6cdf0b69: every step passes except lint, mac-check: lint ^warning: 44, above the ratchet 41 in tools/lint-ratchet.txt. Main at d0cbd4bb gives the same line on the same Mac, with lint its only red step, so the three extra warnings come from main and this PR adds none. Before the rebase, at 618aec3f, lint was at the ratchet (mac-check: lint ^warning: 41 = the ratchet 41 in tools/lint-ratchet.txt).

just gate-gpu-lib at 30f18e63 over its base b3153d77, on the RTX 5090 (BLOOMERY_ARCH=sm_120 BLOOMERY_BOX_LOCAL=1 just gate-gpu-lib), which runs the new test with the rest of bloomery-gpu's library tests:

test result: ok. 240 passed; 0 failed; 0 ignored; 0 measured; 4 filtered out; finished in 5.39s
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 240 filtered out; finished in 1.58s

Gate. hw_geglu_is_the_rule_within_its_bound in crates/gpu/src/gelu.rs, a test gate-gpu-lib runs. At 6cdf0b69 I ran it alone on the RTX 5090 (test gelu::tests::hw_geglu_is_the_rule_within_its_bound ... ok): BLOOMERY_ARCH=sm_120 bash tools/gate.sh --oxide -p bloomery-gpu --release --lib -- --ignored --nocapture gelu::tests::hw_geglu_is_the_rule_within_its_bound.

• Inputs: every gate value i/256 for i in -4096..=4096, plus ±20, ±37.5, ±100, ±1e3, ±1e4 (past where tanh saturates); up values seeded in [-2, 2], every eighth exactly 1.
• Every output is finite, and each is held to a first-order bound of the exact unit in f64 at the rule's f32 constants: the argument's four roundings carried through tanh by 1 - T², libdevice tanhf's 2 ulp, the 1 + t rounding and the two products. The bound is scaled by |0.5 * gate * up|, not by |y|, because 1 + tanh cancels toward zero for large negative gates.
• The output buffer is padded with a guard value past n, and the guard must come back untouched.
• A NaN gate (at -1.61), a NaN up value at a gate of 0.73, and a NaN up value at a saturated gate of -16 give NaN at exactly those three indices.
• n == 0 is refused, and so is n == 9 with one buffer 8 long and the other two 9 long, for each of gate, up and y in turn; each error names enqueue_geglu.

The gate prints how many outputs equal the host rule bit for bit. That count depends on the host's tanhf, so the gate prints it and does not assert it. On the RTX 5090 (BLOOMERY_ARCH=sm_120):

geglu: 8203 values, 0 past their bound, worst error/bound 0.493, 8030 bit for bit the host rule
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 244 filtered out; finished in 6.68s

The same test on the DGX Spark at b689bc93 (BLOOMERY_ARCH=sm_121), whose kernel and rule are the head's:

geglu: 8203 values, 0 past their bound, worst error/bound 0.493, 7996 bit for bit the host rule
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 243 filtered out; finished in 7.13s

Move and add proof. At 0eb4a353, whose kernel and rule are the head's, on the RTX 5090 against its base b3153d77:

• ptx-scan at sm_120 and at sm_86 of generate_ds41, gate_e2e and gate_qwen3vl_tower: every entry of b3153d77 has the same PTX at 0eb4a353 (302, 235 and 251 entries), none is changed or removed, and the one added entry is geglu_tanh. The moved body, vis_gelu_tanh in gate_qwen3vl_tower, has the same MD5 at b3153d77 and 0eb4a353 on both arches (474c92764f801105a737191ba2841d13), 13 registers, no spill and, through the RTX 5090's JIT, no local memory.
• Spill FAIL-first: with geglu_tanh's rows left out of the shape tables, ptx-spill fails on generate_ds41 and gate_e2e (geglu_tanh unpinned spill=0 jit_local=0); with them, all three bins pass. The sm_86 rows come from ptxas, with the stack frame standing in for the 3090 JIT's local bytes.
• The 14 gates gate-paths.tsv routes gelu_tanh.rs to (the MiniCPM tower gate among them, which also owns minicpmv/host.rs) end with the same rc at 0eb4a353 and at b3153d77. None of them passes on this machine, which lacks their model files or cards, so they are not evidence that the vision path is green. gate-gpu-vision-lib, which holds the shared host rule, passes its 8 tests on both sides.

Gates just affected selects and I did not run. tools/affected-gates.sh e98d6c43..618aec3f --no-box, the script behind just affected, selects 114 of 162 gate recipes, through crates/gpu/src/elem.rs, gelu.rs and the vision files. At the head I ran the new test alone; gate-gpu-lib and the owner gates ran at the commits named above. I did not run the others: at 0eb4a353 no existing entry of b3153d77 changed its PTX, and no engine path calls geglu_tanh yet, so by the change-class table the runtime gates are the new test and the moved body's owners.

FAIL-first. Each mutant was a temporary patch, applied alone and reverted after its run. The short-buffer cases, at the head on the RTX 5090: with each of enqueue_geglu's three length checks removed alone, the test fails on that buffer's case (a short gate buffer, a short up buffer, a short y buffer), and the restored tree passes.

The output guard and the saturated-gate NaN case, at b689bc93, whose test had one short-buffer case where the head has three:

Mutant	Red line
the kernel's bound check at i > n instead of i >= n, so it writes y[n]	geglu: output past n must stay untouched
a gate below -10 writes 0 without reading the up value	geglu: a NaN input is NaN at its index alone: NaN at [100, 700], not [0, 100, 700]

The other clauses at 36f2175a, where the kernel is the same and the test lacked the two clauses above:

Mutant	Red line
the kernel returns NaN for every positive gate value	geglu: gate 3.90625e-3 up 1.8667641e0: NaN (the finiteness check)
the kernel multiplies gelu_tanh(up) by the gate (operands swapped)	geglu: values past their bound, 8201 values
the polynomial loses its second factor of x (1 + 0.044715 * x, in gelu.rs)	geglu: values past their bound, 5924 values
a NaN gate becomes a large negative number (g.max(-3e38)) before the rule	geglu: a NaN input is NaN at its index alone
enqueue_geglu drops its n == 0 check	an empty launch: geglu_tanh: Grid.X must be nonzero, the driver's error, not the named refusal

Hardware (if the change is for a machine)

RTX 5090, on the host:

NVIDIA GeForce RTX 5090, 595.99.02, 12.0

DGX Spark, dev container nvidia/cuda:13.3.1-devel-ubuntu24.04:

NVIDIA GB10, 595.84, 12.1

What I ran into

None of this blocks the PR.

• The cc 8.6 row of tools/ref/ptx-shapes.tsv for geglu_tanh comes from ptxas -arch=sm_86 -v and was not read on a 3090. If a reading there differs, the row changes to it (model: EmbeddingGemma and EmbeddingGemma 2 on a side seat #190, open question 1).
• An earlier form of the gate asserted the bit-for-bit count (7996 on the DGX Spark). On the RTX 5090 the count is 8030 with every value inside its bound, because the host's tanhf differs. The count is now printed only.

AI help

AI 코딩 에이전트가 코드를 작성한 후 설명을 썼고, 테스트를 마쳤습니다. diff 리뷰와 PR 초안 리뷰는 제가 직접 했습니다.

[github-actions]

Verdict: MUST_FIX: The new gate duplicates the existing rounding owner.

Must fix

1. Reuse the shared rounding definitions

crates/gpu/src/gelu.rs:43 duplicates rounding::gamma in crates/gpu-gates/src/rounding.rs:17. Its local U also duplicates that module's unit roundoff. Both implement the same f32 error model (R14).
Fixing this now removes two definitions with one test dependency, before other bounds depend on the new copies.

Append to crates/gpu/Cargo.toml:

[dev-dependencies]
bloomery-gpu-gates = { path = "../gpu-gates" }

Replace the test imports and local U/gamma definitions with:

    use super::*;
    use crate::Gpu;
    use bloomery_gpu_gates::rounding::{U, gamma};
    use cuda_core::DeviceBuffer;

Non-blocking

⚠ Supply the change-class proof: crates/gpu-vision/src/gelu_tanh.rs:64 moves a live device body; the elementwise module also gains an entry.
The body has no base/head PTX comparison, pre-pin gate-ptx-spill failure, or owning-gate output for the existing vision callers.
Supply the move/add proofs from AGENTS.md: existing entries unchanged, exactly the new GEGLU entries added, spill FAIL-first, and owning gates green. No timed A/B is required for these classes.

⚠ Isolate the short-buffer cases: crates/gpu/src/gelu.rs:160 makes all three buffers short together.
Deleting any one length check at elem.rs:1994 still passes because the other two return the expected error.
Keep the same gate and add cases with only gate, only up, and only y short. Each dropped check must fail the named-error assertion.

Checked

Duplicates: 12 added or moved function/constant declarations checked; U/gamma duplicate the existing owner. SWIGLU/GEGLU share launch shape but apply different rules. Dryer reported 0 pairs.
Complexity: crapper flagged no functions; no qualifying growth.
Silent failure: 1 max site followed to printed diagnostics only. Empty/short inputs return Shape; NaN inputs propagate through the final multiply to y[i].
Tests: static mutation check: elem.rs:1021 >= to > fails the tail assertion; operand swap at :1029 or lost x at gelu.rs:29 fails the numeric bound; saturated-gate zeroing fails the NaN-index assertion; dropping n == 0 fails the named-error assertion. Individual length-check mutants survive.
Gates: unsafe count 1769 to 1770 has a dated PIN and matches the added block; no existing numeric gate relaxed or removed.
Performance: shared vision arithmetic retains its operation order and inline core; GEGLU has no engine caller and adds no engine-step launches or copies.
Escape catalog: C1–C4 do not match. No ledger lessons are in force.

Notes

Static review only; CUDA gates were not run here. Reported card runs lack the nvidia-smi line, the base/head commit pair, and just gate-* recipe output required for gpu.evidence=true.

_{Card check: needs-gpu. This pull request changes crates/gpu-vision/src/gelu_tanh.rs, crates/gpu-vision/src/minicpmv/host.rs, crates/gpu/src/elem.rs … and carries no gate run on a card (CONTRIBUTING's evidence list); a maintainer adds gpu-verified after one.}

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

Filed outside this PR: #239

[github-actions]

Verdict: CLEAN: No must-fix finding remains.

Non-blocking

⚠ Complete the owner-gate evidence: crates/gpu/src/elem.rs:1117. The new family gate passes, but the PR supplies no passing engine owner-gate run. The base contract's “add (no engine caller yet)” class also requires owning gates of entries engine bins gained; gate_e2e gains this entry. Identify which owners gate-gpu-lib covers and supply the remaining applicable output.

Checked

Duplicates: the moved host rule and extracted device rule have one production owner. Dryer reported 0 pairs. The short swiglu/geglu launch scaffolding shares its module and element view; another abstraction is not justified.

Complexity: crapper flagged no changed function.

Silent failure: no new defaults or ignored Results. Empty and short-buffer cases return named errors; launch_u32 rejects oversized counts; the arithmetic propagates NaNs.

Tests: the new ignored hardware gate checks numeric bounds, finite outputs, the output tail, NaN locations, and named empty/short-buffer refusals. The PR reports FAIL-first failures for the bound-check, operand-order, polynomial, NaN and refusal mutants; the corresponding assertions exist.

Gates: no numeric band relaxed. The unsafe count gains the new block with a dated PIN and reason. The moved host function and constants are identical, and explicit device rounding order is preserved. The PR reports unchanged existing-entry PTX and new-entry spill FAIL-first results.

Adversarial checks: the kernel guards partial blocks; the launcher checks empty/short buffers and narrowing. The test covers saturated negative gates with NaN up values. No engine path calls enqueue_geglu yet.

Escape catalog: C1–C4 do not match the changed code.

Notes

CUDA gates and PTX scans were not rerun here; their results come from the PR body.

_{Card check: gpu-evidence. This pull request changes crates/gpu-vision/src/gelu_tanh.rs, crates/gpu-vision/src/minicpmv/host.rs, crates/gpu/src/elem.rs … and carries a gate run on the author's card, which counts as verified.}

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[juliankang4]

The review is done and the contribution check passed, so I'm merging this.