Skip to content

fix(deepseek): follow pypto-lib 1M-context block tables and bump pypto-lib - #234

Merged
superxf merged 3 commits into
hw-native-sys:mainfrom
zhangqi-chen:feat/deepseek-v4-1m-block-tables
Sep 15, 2026
Merged

superxf merged 3 commits into
hw-native-sys:mainfrom
zhangqi-chen:feat/deepseek-v4-1m-block-tables

Conversation

@zhangqi-chen

@zhangqi-chen zhangqi-chen commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Follow hw-native-sys/pypto-lib#1220, which compiles DeepSeek V4 Flash MTP for 1048576 positions and splits its MoE into decode and prefill modules.

What this changes

  • Submodule 216b497 → 617e165 (pypto-lib#1220 on main). The DeepSeek V4 Flash MTP static logical tables now span 8192 source-token blocks; decode compressed, indexer and compressor-state tables are runtime-width. The other pypto-lib commits between the pins (#1109, #1144, #1179, #1186-#1188, #1190, #1191, #1193, #1195-#1198, #1201-#1203, #1206, #1207, #1216) ride along.
  • Block-table widths (npu_runner.py, task_args.py, cli/main.py):
    • prefill ori / cmp / idx tables: 128 → 8192 blocks
    • prefill HCA / CSA / CSA-inner state tables: 2048 / 4096 / 4096 → 131072 / 262144 / 262144
    • decode ori table: 128 → 8192 blocks
    • decode cmp / idx tables stay at 128 blocks; new decode_{hca,csa,csa_inner}_state_table_blocks (2048 / 4096 / 4096, derived from DEEPSEEK_V4_MAX_SEQ_LEN) size the decode state tables, so per-step host metadata keeps its size
    • the max_model_len checks read the decode CSA state table depth, so serving still caps at 16384 until the decode widths are raised
  • MoE module eviction: add decode_moe / prefill_moe to the DeepSeek V4 import-context eviction list so each executor load re-specializes them for --ep (moe kept for older pins).

Commits

  1. fix(deepseek): follow pypto-lib 1M-context block-table widths
  2. fix(deepseek): evict split decode_moe/prefill_moe kernel modules
  3. deps: bump pypto-lib to 617e165 for DeepSeek V4 1M context

Verification

  • Device (8 cards, DP8/EP8, dsv4-flash-w8a8), serving CI on pypto 747e1e4b: k1-fused completes with the unchanged expected text ("…大典礼的场所…东六宫东") and K=3 passes.
  • Known failure: k1-prefix-cache. The 1202-token prompt (max_model_len=2048, 1024-token chunks) fails on the first prefill dispatch on all eight ranks with 507018 / orch_error_code=5 INVALID_ARGS. The bump crosses pypto-lib #1186/#1203 (CP prefill and prefix cache) as well as #1220, and pypto-lib's own serving CI runs only k1-fused, so the responsible lib change is still being isolated.
  • Unit: tests/unit/model/deepseek 118 passed; the one failure (test_deepseek_mtp_prefill_reads_only_selected_owner_outputs, FakeWorker copy_from() rejects src_offset) also fails on pristine main. Ruff, header, English, and public-docs checks clean.
  • Not run locally: the DSpark 16-card guard. DSpark speculation (K=7): drafter chain, verify, and acceptance #223 noted that pypto-lib past 216b497 (through #1187) deadlocked the DSpark serving dispatch path; this bump crosses that range, so the serving-dspark CI job is the check for it.

@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: d9a3bb95-d73c-481f-b4ac-9bbf0b0d0e5e


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@zhangqi-chen
zhangqi-chen force-pushed the feat/deepseek-v4-1m-block-tables branch 5 times, most recently from 67e35e8 to d1da05d Compare September 15, 2026 06:15
zhangqi-chen and others added 3 commits September 14, 2026 23:16
pypto-lib now compiles DeepSeek V4 Flash MTP for 1048576 positions, so
its static logical tables span 8192 source-token blocks:

- Prefill ori/cmp/idx tables: 128 -> 8192 blocks; prefill HCA / CSA /
  CSA-inner state tables: 2048/4096/4096 -> 131072/262144/262144
- Decode ori table: 128 -> 8192 blocks
- Decode compressed, indexer and state tables are runtime-width in the
  kernels. Serving keeps the decode cmp/idx tables at 128 blocks and adds
  decode_{hca,csa,csa_inner}_state_table_blocks (2048/4096/4096, derived
  from DEEPSEEK_V4_MAX_SEQ_LEN) for the decode state tables, so the
  per-step host metadata stays the same size
- The max_model_len checks read the decode CSA state table depth
pypto-lib splits deepseek_v4_flash_mtp/moe.py into decode_moe.py and
prefill_moe.py, each applying the --ep config override on import. Add
both names to the DeepSeek V4 import-context eviction list so every
executor load re-imports them against the freshly specialized config.
Keep "moe" for pypto-lib revisions that predate the split.
Move the pypto-lib submodule from 216b497 to 617e165 (hw-native-sys/pypto-lib#1220),
which compiles DeepSeek V4 Flash MTP for 1048576 positions and splits its
MoE into decode_moe.py and prefill_moe.py.
@superxf
superxf merged commit be0cd66 into hw-native-sys:main Sep 15, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants