feat(deepseek): add optional MoE expert statistics dump - #185
Draft
high-cloud wants to merge 4 commits into
Draft
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
- Expose online and offline controls for per-layer physical-expert load statistics without changing the disabled runtime path - Reuse fixed resident buffers across prefill, decode, MTP, and fused MTP dispatches, and write routed token counts as JSONL - Add unit coverage and an operator skill for enabling and validating the dump Closes hw-native-sys#181
Bind the MoE statistics buffer explicitly for MTP prefill so the optional dump ABI works in both enabled and disabled modes. Publish MTP initialization mirrors from registered allocation bases because DistributedWorker rejects interior device pointers. Zero the full mirrors before their first upload; a PyPTO offset-copy API will be tracked separately to avoid whole-shard updates in continuous serving.
high-cloud
force-pushed
the
deepseek-moe-stats-dump
branch
from
August 24, 2026 12:38
7977d9b to
4282f44
Compare
Preserve the dynamic prefill TaskArgs path while appending the MoE dump runtime scalar after rebasing onto chunked-prefill support. Carry moe_token_counts through the newly split standalone MTP ABI and update the existing kernel-signature assertions to cover runtime scalars explicitly.
high-cloud
marked this pull request as draft
September 1, 2026 03:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add an optional DeepSeek V4 MoE load-statistics dump that keeps one fixed
kernel ABI whether collection is enabled or disabled. Enabled runs emit
per-layer physical-expert activation and routed-token counts as JSONL; disabled
runs skip buffer clearing, device-to-host copies, and file writes.
Changes
--moe-stats-outputto the unified online/offline CLI and allow embeddedexecutors to enable, redirect, or disable collection at runtime.
for prefill, decode, MTP prefill, standalone MTP decode, and fused MTP decode.
as an unspecialized runtime scalar.
by the current
DistributedWorker; a PyPTO offset-copy API is tracked as thefollow-up needed to replace whole-rank initialization uploads.
output validator.
pypto-libto the fixed-ABI implementation fromAdd: dump DeepSeek V4 MoE expert load statistics pypto-lib#1016.
Verification
/data/yangyaodong/.conda/envs/pypto-lib/bin/python -m pytest tests/unit:208 passed.
python tests/lint/check_headers.py: passed.python tests/lint/check_english_only.py: passed.ruff check --config ruff.toml .: passed.task_20260824_005902_21669804308on physical devices1,3,5,7,9,11,13,15: 32 requests each generated 256 tokens (8192 total),and the JSONL dump contained 215 records across prefill, MTP prefill, and
fused MTP decode.
Closes #181