Qwen3.6-35B-A3B edge runtime: white-boxed prefill, SM110 support, and token-exact speculative decode - #169

Open

LiangSu8899 wants to merge 86 commits into

mainfrom

feat/qwen36-moe-edge-runtime

Commits on Jul 29, 2026

Add Qwen3.6-35B-A3B text frontend
LiangSu8899
committed
cda967a
Harden Qwen3.6 runtime contracts
LiangSu8899
committed
1db9b46

Commits on Jul 30, 2026

Add Qwen3.6-MoE edge checkpoint utilities
LiangSu8899
committed
c92dc19
Split qwen3_5_moe kernels into portability tiers
LiangSu8899
committed
12e34a7
Simulate a two-tier expert cache in the route trace
LiangSu8899
committed
1af9321
Align routed-expert blocks for direct I/O
LiangSu8899
committed
7525d8f
Decode packed E2M1 without requiring cuda_fp4.h
LiangSu8899
committed
2a3c434
Record which expert-cache policy actually wins
LiangSu8899
committed
bab295a
Score expert quantization against real routed activations
LiangSu8899
committed
ed687c1
Give the INT4 expert block a two-level scale
LiangSu8899
committed
9447bd4
Model warm-started cache and cold prefill cost
LiangSu8899
committed
c19d728
Validate the startup expert set on held-out prompts
LiangSu8899
committed
eba1014
Add cross-architecture parity for the gated kernels
LiangSu8899
committed
153be4b
Add a bounded streaming cache for routed-expert blocks
LiangSu8899
committed
9db0327
Make the page-cache bypass switchable so it can be measured
LiangSu8899
committed
edc28ac
Keep the router's order when a request is deduplicated
LiangSu8899
committed
182fb85
Let the validation persist its traces
LiangSu8899
committed
2e70e82
Expose a block's components and its global scales
LiangSu8899
committed
a730abc
Decode a streamed expert block into bf16
LiangSu8899
committed
ed77aef
Serve the routed experts from storage
LiangSu8899
committed
d50ad73
Load the tokenizer only when something asks for it
LiangSu8899
committed
9f9fe5a
Let a configuration state which kernels it calls
LiangSu8899
committed
69e1252
Pass stream_experts to the loader, and give lm_head a portable path
LiangSu8899
committed
eabfc82
Fix three defects in the streamed-expert assembly
LiangSu8899
committed
2ad746a
Check the vendored attention kernel actually computes, and fall back
LiangSu8899
committed
5efdb40
Treat a missing FA2 as a fallback rather than an error
LiangSu8899
committed
00c7b74
Document why attention differs between targets
LiangSu8899
committed
d2ebdcc
Let the text prefill run where the vendored FA2 is absent
LiangSu8899
committed
7a12744
Take the bank conflict out of the weight-only 4-bit GEMVs
LiangSu8899
committed
ee73fca
Give each warp several rows so the 4-bit GEMVs have loads in flight
LiangSu8899
committed
c955c59
Tune rows per warp per entry point, and drop the KV broadcast copy
LiangSu8899
committed
6f60dc2
Size the GEMV block to the part, not to the card it was written for
LiangSu8899
committed
b206be1
Load and run the multi-token-prediction head
LiangSu8899
committed
1ca7849
Draft, verify and rewind a speculative window
LiangSu8899
committed
2d61232
Give the verify block a 4-bit path, and measure that it is not the problem
LiangSu8899
committed
22cb219
Capture the speculative window, and stop quantising the draft head
LiangSu8899
committed
0c6d31c
Take the rollback snapshots before the state they snapshot is overwritten
LiangSu8899
committed
c2178cc
Group prefill's routed tokens by expert instead of re-reading the weight
LiangSu8899
committed
2a4c5ff

Commits on Jul 31, 2026

Hand prefill's dense projections to cuBLASLt, and hoist the gate out of the loop
LiangSu8899
committed
35127ca
Serve every routed expert of a prefill layer in one grouped GEMM
LiangSu8899
committed
80d7009
Unpermute the grouped MoE without atomics
LiangSu8899
committed
4c6cdda
Stop forcing prefill attention onto the math backend
LiangSu8899
committed
4a058b3
Take the atomics out of the grouped activation quantiser
LiangSu8899
committed
8dc4ac4
Gate and quantise the grouped MoE's intermediate in one pass
LiangSu8899
committed
d45a3d8
Stop upcasting the KV cache to fp32 on every decode step
LiangSu8899
committed
1d30ed8
Record what chunked prefill's attention can and cannot use
LiangSu8899
committed
b19fc75
Gather the routing permutation inside the quantiser, and cache what is constant
LiangSu8899
committed
cc662b6
Take the WY front matter off the host
LiangSu8899
committed
7697283
Route a prefill's experts in kernels, not tensor ops
LiangSu8899
committed
faf08a9
Fuse the residual adds and the MoE tail into their kernels
LiangSu8899
committed
213adf7
Cover the prefill routing kernel
LiangSu8899
committed
5fd2967
Scan the routing histogram across the block, not along one thread
LiangSu8899
committed
0364058
Build FA2 on sm_110, where the chunked window needed it
LiangSu8899
committed
54ff8db
Cover the rest of the prefill kernels
LiangSu8899
committed
b0167d5
Read the conv1d input once instead of once per output
LiangSu8899
committed
0c38caa
Cover the row-blocked conv1d
LiangSu8899
committed
0dde39f
Let the chunked conv read its history instead of concatenating it
LiangSu8899
committed
a61e874
Give the gate-and-quantise a lane per scale-factor group
LiangSu8899
committed
bdf5ddb

Commits on Aug 1, 2026

Commits on Aug 6, 2026