Revert "perf(deepseek): reduce fused MTP decode host transfers" - #186
Open
ndleslx wants to merge 1 commit into
Open
Revert "perf(deepseek): reduce fused MTP decode host transfers"#186ndleslx wants to merge 1 commit into
ndleslx wants to merge 1 commit into
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ndleslx
force-pushed
the
revert/pr144-fused-mtp-decode
branch
from
August 25, 2026 03:50
e308625 to
19d66be
Compare
ndleslx
force-pushed
the
revert/pr144-fused-mtp-decode
branch
from
August 26, 2026 03:44
19d66be to
38d471f
Compare
This reverts commit 0ae4fd85e00ea4f89f6eed52ea37cbc3b0eb0f168, re-resolved against current main so hw-native-sys#132's chunked-prefill and K=1/K>1 MTP ABI changes are kept while hw-native-sys#144's device-residency is removed: TaskArgs slots are host-shared again, the decode-metadata residency machinery (_materialize_decode_device_metadata and friends) is gone, and hw-native-sys#144's two device-residency tests are dropped. The change broke the DeepSeek V4 accuracy guard on main: the merge run of hw-native-sys#144 failed test_deepseek_v4_http_completion_matches_expected_text [k1-prefix-cache] with a generated-text mismatch, while the previous merge run on main (hw-native-sys#170, the same morning) was green — both runs on the identical pypto revision. Restore the pre-hw-native-sys#144 decode path until the transfer reduction lands with the guard passing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reverts #144.
#144 (
perf(deepseek): reduce fused MTP decode host transfers, commit0ae4fd85) broke the DeepSeek V4 accuracy guard on main:(32468071303)
failed
unit-testsontests/test_deepseek_v4_http_completion_matches_expected_text[k1-prefix-cache]— an accuracy mismatch, the generated text diverging from the expected
completion mid-generation.
32462006878)
was green 80 minutes earlier, and nothing landed in between.
pypto under test: 5b15048e836c574238acde135466740c10bf5239appears in each log — so ptoas andthe pto-isa commit (read from that same checkout) were the same as well. The
toolchain is held constant; the only delta between green and red is perf(deepseek): reduce fused MTP decode host transfers #144
itself.
That is the path #144 changed: the fused MTP decode in
npu_runner.py/task_args.pynow keeps outputs device-resident, and thek1-prefix-cachevariant is exactly the configuration that exercises it with acached prefix.
Verified locally: the reverted tree passes all 82 tests in
tests/unit/model/deepseek/test_model_components.py, plus the repo's lintchecks. The full
unit-testsjob — including that DeepSeek guard — runs on thisPR and should confirm main goes green again.
The perf idea is worth relanding once it can pass the accuracy guard.