fix(deepseek): follow pypto-lib 1M-context block tables and bump pypto-lib - #234
Merged
superxf merged 3 commits intoSep 15, 2026
Merged
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
zhangqi-chen
force-pushed
the
feat/deepseek-v4-1m-block-tables
branch
5 times, most recently
from
September 15, 2026 06:15
67e35e8 to
d1da05d
Compare
pypto-lib now compiles DeepSeek V4 Flash MTP for 1048576 positions, so
its static logical tables span 8192 source-token blocks:
- Prefill ori/cmp/idx tables: 128 -> 8192 blocks; prefill HCA / CSA /
CSA-inner state tables: 2048/4096/4096 -> 131072/262144/262144
- Decode ori table: 128 -> 8192 blocks
- Decode compressed, indexer and state tables are runtime-width in the
kernels. Serving keeps the decode cmp/idx tables at 128 blocks and adds
decode_{hca,csa,csa_inner}_state_table_blocks (2048/4096/4096, derived
from DEEPSEEK_V4_MAX_SEQ_LEN) for the decode state tables, so the
per-step host metadata stays the same size
- The max_model_len checks read the decode CSA state table depth
pypto-lib splits deepseek_v4_flash_mtp/moe.py into decode_moe.py and prefill_moe.py, each applying the --ep config override on import. Add both names to the DeepSeek V4 import-context eviction list so every executor load re-imports them against the freshly specialized config. Keep "moe" for pypto-lib revisions that predate the split.
Move the pypto-lib submodule from 216b497 to 617e165 (hw-native-sys/pypto-lib#1220), which compiles DeepSeek V4 Flash MTP for 1048576 positions and splits its MoE into decode_moe.py and prefill_moe.py.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow hw-native-sys/pypto-lib#1220, which compiles DeepSeek V4 Flash MTP for 1048576 positions and splits its MoE into decode and prefill modules.
What this changes
216b497 → 617e165(pypto-lib#1220 on main). The DeepSeek V4 Flash MTP static logical tables now span 8192 source-token blocks; decode compressed, indexer and compressor-state tables are runtime-width. The other pypto-lib commits between the pins (#1109, #1144, #1179, #1186-#1188, #1190, #1191, #1193, #1195-#1198, #1201-#1203, #1206, #1207, #1216) ride along.npu_runner.py,task_args.py,cli/main.py):decode_{hca,csa,csa_inner}_state_table_blocks(2048 / 4096 / 4096, derived fromDEEPSEEK_V4_MAX_SEQ_LEN) size the decode state tables, so per-step host metadata keeps its sizemax_model_lenchecks read the decode CSA state table depth, so serving still caps at 16384 until the decode widths are raiseddecode_moe/prefill_moeto the DeepSeek V4 import-context eviction list so each executor load re-specializes them for--ep(moekept for older pins).Commits
fix(deepseek): follow pypto-lib 1M-context block-table widthsfix(deepseek): evict split decode_moe/prefill_moe kernel modulesdeps: bump pypto-lib to 617e165 for DeepSeek V4 1M contextVerification
747e1e4b:k1-fusedcompletes with the unchanged expected text ("…大典礼的场所…东六宫东") andK=3passes.k1-prefix-cache. The 1202-token prompt (max_model_len=2048, 1024-token chunks) fails on the first prefill dispatch on all eight ranks with507018/orch_error_code=5 INVALID_ARGS. The bump crosses pypto-lib #1186/#1203 (CP prefill and prefix cache) as well as #1220, and pypto-lib's own serving CI runs onlyk1-fused, so the responsible lib change is still being isolated.tests/unit/model/deepseek118 passed; the one failure (test_deepseek_mtp_prefill_reads_only_selected_owner_outputs, FakeWorkercopy_from()rejectssrc_offset) also fails on pristine main. Ruff, header, English, and public-docs checks clean.serving-dsparkCI job is the check for it.