[NPU] Enable Qwen3-VL GRPO (single- & multi-turn) on Ascend

[Meihan-chen]

Summary

Brings up Qwen3-VL-8B-Instruct GRPO on Ascend NPU for both the single-turn (examples/geo3k_vlm) and multi-turn (examples/geo3k_vlm_multi_turn) examples, on vllm-ascend rollout + Megatron-Bridge/MindSpeed training.

Base VLM fixes (apply to both single- & multi-turn)

• Weight load: register MindSpeedTEColumnParallelLinear in Bridge AutoMapping (megatron-bridge.patch).
• Forward — nvtx: replace raw torch.cuda.nvtx.* with megatron.core.utils.nvtx_range_* (no CUDA NVTX on NPU) (megatron-bridge.patch).
• Forward — THD mrope: preprocess_packed_seqs integer-indexed an int32 mask → mrope positions collapse (ppl ~13000, abs_diff ≈ 9); build the mask as bool (megatron-bridge.patch).
• Vision attention: MindSpeed varlen kernel forced a causal mask despite attn_mask_type=no_mask, breaking the bidirectional vision tower (image abs_diff ≈ 0.8); honor no_mask (mindspeed.patch).
• Weight sync: packed_broadcast_producer read source tensors on the packing stream before the caller stream finished; add wait_stream(source_stream) (vllm-ascend.patch).

Multi-turn fixes

• Prefix-stability / token desync: re-rendering the whole conversation each turn triggers a tokenizer BPE re-merge at the assistant\n seam, shifting the token stream away from what vLLM actually generated. Fixed by aligning rollout.py to upstream [Bugfix][Rollout] Fix Geo3K VLM multi-turn rollout #341 — render once + observation-suffix delta.
• The rewritten rollout imports only helpers already present on the ascend branch (GenerateState, _build_inference_sampling_params, _coerce_flat_int_token_ids, _mm_render_response_to_generate_body) and inlines response/routing parsing (_parse_choice, _decode_routing_metadata), so it no longer depends on the vime-only helpers and vime/rollout/vllm_rollout.py is untouched.

[read-the-docs-community]

Documentation build overview

📚 vime | 🛠️ Build #33614259 | 📁 Comparing 2ee0d79 against latest (5532763)

  🔍 Preview build  

43 files changed · + 2 added · ± 27 modified · - 14 deleted

+ Added

• get_started/NPU.html
• _examples_synced/search-r1/README.html

± Modified

• genindex.html
• index.html
• search.html
• advanced/arch-support-beyond-megatron.html
• advanced/fault-tolerance.html
• advanced/megatron-config.html
• advanced/pd-disaggregation.html
• advanced/reproducibility.html
• advanced/speculative-decoding.html
• advanced/vllm-config.html
• and 17 more...

- Deleted

• advanced/delta-weight-sync.html
• advanced/external-rollout-engines.html
• advanced/low-precision.html
• advanced/observability.html
• examples/deepseek-r1.html
• examples/glm4-9B.html
• examples/glm4.7-30B-A3B.html
• examples/glm4.7-355B-A32B.html
• examples/glm5.2-744B-A40B.html
• examples/qwen3-4b-base-openhermes.html
• and 4 more...

[gemini-code-assist]

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

[Meihan-chen]

Qwen3-VL-8B-Instruct-multi-turn-NPU

Qwen3-VL-8B-Instruct-multi-turn-GPU

[Meihan-chen]

Qwen3-VL-8B-Instruct-single-turn vs multi-turn

[floatlibai]

The main branch seems to have a refactor #341 , maybe you can have a look.

[floatlibai]

This attention_mask line can be removed.

[Meihan-chen]

thanks，done

[Meihan-chen]

The main branch seems to have a refactor #341 , maybe you can have a look.

#341 fixes the same two multi-turn issues as our earlier in-house patch, using the same approach — render once, keep the generated token ids as the canonical history, and append only the observation suffix delta between turns.

The only difference was code structure (our hand-rolled generate() vs upstream’s _Geo3kRollout class). This PR adopts #341 verbatim so the file matches upstream, with a single required shim: this Ascend branch’s Sample has no append_response_tokens(), so _Geo3kRollout._append_response() maintains tokens/loss_mask/rollout_log_probs/response_length directly. diff vs upstream = only that method + its 3 call-sites.

[CalvinXKY]

LGTM. Nit: PR description is slightly stale — it still says we restore 3 helpers in vime/rollout/vllm_rollout.py deleted by #178, but this diff does not touch that file. The multi-turn rollout now imports fewer helpers (_coerce_flat_int_token_ids / _mm_render_response_to_generate_body already present on ascend) and aligns with the #341 “render once + observation suffix” approach. Worth updating the “Multi-turn fixes” section so it matches the final code.

[Meihan-chen]

LGTM. Nit: PR description is slightly stale — it still says we restore 3 helpers in vime/rollout/vllm_rollout.py deleted by #178, but this diff does not touch that file. The multi-turn rollout now imports fewer helpers (_coerce_flat_int_token_ids / _mm_render_response_to_generate_body already present on ascend) and aligns with the #341 “render once + observation suffix” approach. Worth updating the “Multi-turn fixes” section so it matches the final code.LGTM。

Good catch — updated. Reworded the alignment note accordingly.