fix(qwen): dispatch decode with runtime batch

[Little-oil]

Summary

• update the pypto-lib submodule to Add: use PyPTO paged attention in Qwen3 decode pypto-lib#1031
• dispatch Qwen decode with runtime-batch prefix views over max-capacity buffers
• stop replicating row 0 into inactive decode rows
• cover inactive-row preservation and host/device runtime-batch views with unit tests

Root cause

The Qwen runner treated decode as fixed-batch. When chunked prefill produced a partial decode batch, it copied row 0's token, sequence length, block table, and slot mapping into every inactive row and launched all max-batch rows.

The decode ABI is runtime-batch dynamic. With native paged attention, each row appends K/V in parallel, so the replicated rows made several lanes write the same physical KV slot concurrently. Under a B16, 3338-token long-context workload this could silently produce all-zero output streams.

This change retains max-batch storage but narrows every batch-shaped dispatch argument to actual_batch, including worker-resident DeviceTensor logits and next-hidden buffers used by device sampling.

Fixes #201

Validation

• python -m pytest -q tests/unit/model/qwen: 31 passed
• header check: passed
• English-only check: passed
• Ruff on changed files: passed
• git diff --check: passed

NPU long-context regression (a2a3):

• Qwen3-14B
• 16 concurrent identical 3338-token prompts
• 128 device-greedy output tokens per request
• max sequence length 4096, chunked prefill on, prefix cache off
• 16/16 identical 128-token streams
• no zero or invalid stream
• 2048 aggregate output tokens in 106.96 s (19.1 tok/s end-to-end)

Qwen performance comparison

PA is fused into the all-layer decode_fwd, so the serving trace does not expose a representative standalone PA event. The comparison therefore reports directly measured fused-Qwen and end-to-end durations rather than derived PA throughput.

Device	Measured scope	CCE baseline	PyPTO PA	Duration delta
Card 3	steady B16 decode_fwd mean	39.096 ms	38.325 ms	-1.97%
Card 3	full Qwen generation window	108.756 s	108.524 s	-0.21%
Card 7	steady B16 decode_fwd mean	38.790 ms	39.374 ms	+1.50%
Card 7	full Qwen generation window	107.635 s	106.992 s	-0.60%

Negative delta means the PyPTO version completed faster. Across the two paired cards, the mean duration changed by approximately -0.24% for fused decode_fwd and -0.40% for the full Qwen generation window, which is performance parity.

Dependency

This PR points the submodule at the merged hw-native-sys/pypto-lib#1031 commit da91587.

Scope

The current CI run passes both Qwen3 guards. Its only failure is the later DeepSeek V4 MTP stage: the pre-existing DeepSeek HC head uses pl.create_tensor(..., init_value=0), which current PyPTO no longer supports. That line is identical at the base and updated pypto-lib revisions, and this branch contains no DeepSeek source changes.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7624e1ad-3078-48cc-a942-f752f0cc2cff

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}