feat(moe): FUSED3=1 opt-in AVX2 expert matmul (olmoe) — rebase of #1024 by @outtodata

[JustVugg]

This is @outtodata's #1024, rebased onto current dev by the maintainers — the commit is theirs and the authorship is preserved; we only resolved the rebase conflict (dev's OLMOE_TESTING hook and the fused_simd.h include coexist — both kept) and re-verified.

Why we carried it instead of waiting

We are executing a performance roadmap across the engines (the #1071–#1077 hoist series, the K1 planar layout in #1079), and the next CPU item is exactly what this PR implements on olmoe: fused gate/up with vectorized activation quantization, bit-identical, opt-in. Having this reviewed and in-tree gives the colibri.c fusion a reference to copy instead of a second invented shape. It stopped being "nice to have" and became sequencing-critical — so we did the rebase rather than let it wait.

Re-verification on the rebased branch (this machine, i7-1355U AVX2+VNNI)

i8_O1024_I2048: v1_ms=0.0443 v2_ms=0.0349 speedup=1.267
i8_O2048_I1024: v1_ms=0.0401 v2_ms=0.0315 speedup=1.272
v3_quant_bitexact=yes
v3_output_bitidentical_v2=yes
v3_expert: v2_3calls_ms=0.1019 v3_pair+down_ms=0.0837 speedup=1.217

The author's own bench, unmodified: quantization bit-exact, output bit-identical, 1.22–1.27× here (the PR's original −40% was measured on their hardware). FUSED3 remains off by default — shipping behavior unchanged.

Closes #1024 — all credit to @outtodata; if you'd rather carry it yourself after all, say the word and we'll close this one instead.

🤖 Generated with Claude Code