fix(deepseek): keep decode metadata host-shared

[high-cloud]

Summary

Remove the fused MTP-1 decode metadata path that serialized every ping-pong slot behind the previous dispatch.

The current main branch materializes otherwise dynamic block metadata as device-resident buffers, copies it outside the dispatch, and waits for a global metadata predecessor before the next update. Under GBS32 this collapses the intended depth-two pipeline into an approximately 198 ms cadence. This change restores the TaskArgs-owned host-shared ping-pong metadata and passes those buffers directly to the fused decode program.

It also copies MTP initialization pools from each allocation base. The current DistributedWorker.copy_to contract reconstructs ownership from the allocation base and does not accept an interior row pointer.

The previously proposed write-only output residency and two-slot materialization are already present on main; this rewritten PR contains only the remaining Serving-side fixes.

Runtime dependencies:

• Fix: align full-rank chip run activation simpler#1967
• fix(runtime): group full-rank chip dispatches pypto#2490

Current upstream PyPTO wheel-build prerequisite for Serving NPU CI: hw-native-sys/pypto#2491.

Tracking issue: #179. This PR intentionally does not close it because a low-frequency random-rank completion/fence tail remains under investigation.

Changes

• Keep fused decode block metadata in each host-shared TaskArgs ping-pong slot.
• Remove the global metadata predecessor and run-external device metadata copies.
• Copy MTP initialization buffers from allocation-base host/device shards.
• Update DeepSeek component tests for the restored metadata placement and allocation-base copy contract.

Verification

• python -m pytest tests/unit/model/deepseek/test_model_components.py -q: 84 passed.
• Four-device EP4/TP4, 5 warmups + 1000 measured rounds, async depth 2: task exit 0; runner-start skew p95/max 0.058/0.201 ms; runner-to-validate gap max 0.067 ms; validate max 3.499 ms.
• Production DeepSeek V4 Serving MTP-1, GBS32, DP8/EP8, 32 concurrent requests x 256 output tokens, same-run verbose SA_PROFILE plus per-process host STRACE:
  • task exit 0; all 8192/8192 output tokens returned
  • batch elapsed 19.189 s; throughput 426.909 tok/s; effective TPOT 74.957 ms
  • runner-start skew p50/p95/p99/max 0.069/0.096/0.281/1.089 ms; no >5 ms late starts
  • runner-to-validate gap p95/max 0.017/0.052 ms
  • validate p95/max 0.182/0.227 ms
  • combined swimlane validated with 143 decode steps and 1144 rank roots

Seven of 143 profiled steps still had runner-end skew above 5 ms (maximum 7.420 ms), on changing physical devices. The acceptance run disabled device-phase markers, so it establishes that these tails remain inside runner_run, but does not by itself attribute every one to native_fence_wait.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 42f67063-0a07-4bd2-91e3-e0caf26cbdd4

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}