perf(deepseek): submit decode asynchronously

[high-cloud]

Summary

• replace the blocking DeepSeek V4 fused decode launch with PyPTO submit() and defer handle.result() to the output/reclaim lane
• retain mutable host bindings and transient device uploads until asynchronous completion, with idempotent wait and cleanup behavior
• make all large host output bindings slot-local so the next prepared decode cannot overwrite an in-flight invocation
• teach the DSV4 profiling workflow to pair .worker_submit and .worker_wait spans and report submit-to-completion timing

Closes #148.

Why

Serving already separated prepare, dispatch, and reclaim, but the device lane still called the blocking PyPTO run() API. Now that PyPTO exposes asynchronous handles, serving can submit the next prepared decode while the output lane waits for the previous invocation and publishes its tokens.

The handle lifetime also changes the binding contract: every mutable Host Out buffer must remain unique and alive until completion, even when reclaim does not read that buffer. The two decode snapshots therefore own separate main and MTP hidden/logit outputs.

Validation

• python -m pytest tests/unit — 144 passed
• changed-file Ruff, header check, English-only check, and git diff --check passed
• real DeepSeek V4 HTTP profile completed with 20 output tokens on devices 0,2,4,6,8,10,12,14
  • HTTP 200, 20 completion tokens
  • MTP acceptance 10/11
  • all three L3 programs hit the serving compile cache
  • fused decode submit-to-completion steady mean: 60.823 ms/iteration
  • Simpler host STRACE critical-rank steady mean: 34.830 ms/iteration

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f86558df-fe8b-4d95-80ec-f1061b455ccf

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

DeepSeek V4 fused MTP decode now submits asynchronous L3 work and reclaims outputs after completion. Slot-local buffers preserve in-flight results. Profiling accepts asynchronous worker spans and reports submit-to-completion timing.

Changes

Asynchronous decode pipeline

Layer / File(s)	Summary
Async dispatch ownership contract pypto_serving/model/common/executor/..., pypto_serving/model/deepseek/npu_runner.py	The executor and runner define pending completion/reclaim tickets. _submit_l3 tracks uploaded tensors and releases them exactly once after completion.
Slot-local fused MTP execution pypto_serving/model/deepseek/npu_runner.py, pypto_serving/serving/server/serving_worker.py, tests/unit/model/deepseek/test_model_components.py	Fused MTP decode submits asynchronously and the reclaim lane waits before reading outputs. Ping-pong slots own independent output tensors. Tests cover deferred completion, cleanup, failures, and slot isolation.
Async lifecycle profiling .agents/skills/profile-dsv4-serving-strace/...	Profile analysis pairs worker submission and wait spans, measures submit-to-completion time, records the timing source, and accepts blocking compatibility spans. The launch command sets NPU memory utilization to 0.99.

Estimated code review effort: 4 (Complex) | ~45 minutes

Mergeability Score: 🟡 Moderate · up to 6972d

The change adds asynchronous decode timing analysis, but the current analyzer can pair overlapping spans and can label fallback blocking timings as submit-to-completion. This can produce incorrect performance reports, so merge should wait for these bounded observability issues to be fixed or explicitly accepted.

Sequence Diagram(s)

sequenceDiagram
  participant DeviceLane
  participant DeepSeekV4Runner
  participant L3Worker
  participant ReclaimLane
  DeviceLane->>DeepSeekV4Runner: submit prepared fused MTP decode
  DeepSeekV4Runner->>L3Worker: submit L3 work and retain slot bindings
  L3Worker-->>DeviceLane: return pending completion handle
  ReclaimLane->>L3Worker: wait for completion
  L3Worker-->>ReclaimLane: complete device work
  ReclaimLane->>DeepSeekV4Runner: reclaim and read outputs

Loading

Possibly related PRs

• hw-native-sys/pypto-serving#147: Extends the asynchronous DeepSeek decode dispatch and reclaim pipeline.
• hw-native-sys/pypto-serving#89: Introduced the DeepSeek V4 MTP decode path extended here.
• hw-native-sys/pypto-serving#125: Introduced the profiling skill updated for asynchronous lifecycle spans.

Poem

A rabbit submits work with a hop,
Then waits till the device says stop.
Slots keep outputs apart,
Cleanup runs once by design,
And profiles now measure the line.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 32.14% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes implement asynchronous submission, handle retention, deferred completion, cleanup, output safety, and serving-lane integration requested by issue #148.
Out of Scope Changes check	✅ Passed	All changes support asynchronous DeepSeek decode serving, its profiling workflow, documentation, or related tests.
Description check	✅ Passed	The description clearly explains the asynchronous decode submission, resource lifetime changes, slot-local outputs, profiling updates, and validation.
Title check	✅ Passed	The title clearly and concisely identifies the main change: asynchronous DeepSeek decode submission.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.agents/skills/profile-dsv4-serving-strace/scripts/analyze_profile.py:
- Line 358: Update the report formatting around the fused decode steady-mean
output to choose its metric label from
summary["serving_profile"]["decode_timing_source"], distinguishing
submit-to-completion from the blocking_kernel_span fallback. Keep the existing
fused_serving mean value and formatting unchanged while ensuring the Markdown
label accurately reflects the timing source.
- Around line 123-128: Update the async kernel lifecycle validation in the
submits/waits loop to require each wait span’s start timestamp to be at or after
the paired submit span’s end (`submit["ts"] + submit["dur"]`). Raise the
existing invalid-lifecycle RuntimeError when this condition fails, before
appending to kernel_durations_ms.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c91456b1-f6b4-4a34-b9c2-0d1c029f5f83

📥 Commits

Reviewing files that changed from the base of the PR and between 34bf409 and 6972da8.

📒 Files selected for processing (8)

• .agents/skills/profile-dsv4-serving-strace/SKILL.md
• .agents/skills/profile-dsv4-serving-strace/scripts/analyze_profile.py
• .agents/skills/profile-dsv4-serving-strace/scripts/run_profile.py
• pypto_serving/model/common/executor/executor.py
• pypto_serving/model/common/executor/pypto_executor.py
• pypto_serving/model/deepseek/npu_runner.py
• pypto_serving/serving/server/serving_worker.py
• tests/unit/model/deepseek/test_model_components.py