Add: use MXFP8 and MXFP4 in DeepSeek V4 Pro - #911
Draft
yanghaoran29 wants to merge 1 commit into
Draft
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
yanghaoran29
force-pushed
the
feat/prototype-mxfp8-shared-expert-down-projection
branch
2 times, most recently
from
August 7, 2026 07:39
b975474 to
ac3e17d
Compare
yanghaoran29
force-pushed
the
feat/prototype-mxfp8-shared-expert-down-projection
branch
from
August 16, 2026 13:55
ac3e17d to
eed5080
Compare
- Add dynamic MXFP8 quantization to dense projections and shared experts - Store routed expert weights as MXFP4 and materialize converted MXFP8 values and scales through GM before matmul - Wire packed MX formats through decode, prefill, layer, MTP, and forward entries with bounded full-forward smoke fixtures - Avoid shared-memory copies for immutable resident golden inputs - Document the separate quant_mx and matmul_mx task limitation
yanghaoran29
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
gate, and shared-expert projections.
and materialize its values/scales through GM before
matmul_mx.entries, and avoid duplicate shared-memory copies for large immutable
resident golden inputs.
quant_mxandmatmul_mxin separate tasks. Putting both in onemixed InCore task is intentionally not supported by this PR.
Test environment
72d9b9f(backup/pr-2372-ai-comments)3165cc80cefc9agreater than
0.1% / 0.3% / 0.5% / 1% / 3% / 5% / 10%.fatal.
A5 operator results
Integer-output exact mismatch rates: gate
x_norm0.003488%, decodeindex-cache 0.000095%, decode top-k 0.006104%, prefill index-cache 0.000191%,
and prefill top-k 0.012207%.
The bounded full-forward runtime smoke passed before the final rebase: decode
with five layers and prefill with three layers. Full 61-layer fixtures exceed
the available host/device memory on this test machine. After rebasing onto
37d6115,tests/golden/test_runner.pypasses (74 tests) and all changedPython sources compile. The rebased full-forward compile reaches the unchanged
upstream
lm_head_greedy_sample, where PTOAS 0.58 rejectssort32without anexplicit temporary; no MX operator fails before that point.