[NPU] refactor(ci): device-aware test harness for GPU/NPU + Qwen3 NPU smoke tests

[Meihan-chen]

What

Introduce a hardware-platform abstraction so a single tests/test_*.py runs on both
GPU and NPU. execute_train() dispatches by device — VIME_TEST_DEVICE (set by CI),
else is_npu(). The GPU/CUDA path is byte-identical to upstream and needs no config;
non-CUDA delegates to a vime-owned launcher.

Part of the NPU CI effort — RFC #315.

Changes

• vime/utils/external_utils/platforms.py (new) — Platform registry: Ray resource
  args, device runtime env, checkpoint mode, unsupported features, and detection. Adding
  an accelerator (XPU/TPU/ROCm) is one register(Platform(...)); the resolver,
  launcher, and seam stay platform-agnostic.
• command_utils.py — platform seam in execute_train (non-CUDA → launch_commands);
  convert_checkpoint no-ops on bridge platforms; removed the dead execute_train_npu.
• tests/test_qwen3_4B_npu.py, tests/test_qwen3_30B_A3B_npu.py — NPU GRPO smoke
  tests (dense npu-8, MoE npu-16) via U.execute_train, faithful to the existing
  run-qwen3-*-npu.sh parameters.

[read-the-docs-community]

Documentation build overview

📚 vime | 🛠️ Build #33631692 | 📁 Comparing 633d2a2 against latest (e0ca828)

  🔍 Preview build  

47 files changed · + 3 added · ± 27 modified · - 17 deleted

+ Added

• get_started/NPU.html
• _examples_synced/retool/README.html
• _examples_synced/search-r1/README.html

± Modified

• genindex.html
• index.html
• search.html
• advanced/arch-support-beyond-megatron.html
• advanced/fault-tolerance.html
• advanced/megatron-config.html
• advanced/pd-disaggregation.html
• advanced/reproducibility.html
• advanced/speculative-decoding.html
• advanced/vllm-config.html
• and 17 more...

- Deleted

• advanced/delta-weight-sync.html
• advanced/external-rollout-engines.html
• advanced/low-precision.html
• advanced/observability.html
• examples/deepseek-r1.html
• examples/gemma4.html
• examples/glm4-9B.html
• examples/glm4.7-30B-A3B.html
• examples/glm4.7-355B-A32B.html
• examples/glm5.2-744B-A40B.html
• and 7 more...

[gemini-code-assist]

Code Review

This pull request introduces NPU test configurations for Qwen3 models and refactors the hardware-platform abstraction into a platform-agnostic registry to support both CUDA and NPU accelerators. The review feedback highlights a potential crash in NPU detection if a RuntimeError is raised, recommends using shlex.quote to safely escape environment variables and JSON strings in shell commands, and points out a potential typo or redundant argument (--kl-coef vs --kl-loss-coef) in the GRPO configuration for the Qwen3 4B NPU test.

[gemini-code-assist]

The is_npu() function in vime/utils/common.py explicitly raises a RuntimeError if torch_npu is installed but no NPU device is available or visible. Since _detect_npu() only catches ImportError, any such RuntimeError will propagate and crash current_platform(), preventing fallback to CUDA or CPU even on non-NPU systems where the package might be present.

We should catch RuntimeError as well to ensure robust platform detection.

Suggested change

	def _detect_npu() -> bool:
	try:
	from vime.utils.common import is_npu
	except ImportError:
	return False
	return is_npu()
	def _detect_npu() -> bool:
	try:
	from vime.utils.common import is_npu
	return is_npu()
	except (ImportError, RuntimeError):
	return False

[gemini-code-assist]

Import shlex to safely escape environment variables and JSON strings when constructing shell commands.

Suggested change

	import json
	import os
	from collections.abc import Callable
	from dataclasses import dataclass, field
	import json
	import os
	import shlex
	from collections.abc import Callable
	from dataclasses import dataclass, field

[gemini-code-assist]

Use shlex.quote to safely escape environment variable values when constructing the export commands, preventing potential shell injection or parsing issues if values contain special characters.

Suggested change

	exports = "".join(f'export {k}="{v}" && ' for k, v in all_env.items())
	exports = "".join(f"export {k}={shlex.quote(v)} && " for k, v in all_env.items())

[gemini-code-assist]

Use shlex.quote to safely escape the JSON string for --runtime-env-json. This avoids syntax errors or shell parsing issues if any environment variable or argument contains single quotes or other special characters.

Suggested change

	cmds.append(
	f"export no_proxy=127.0.0.1 && export PYTHONUNBUFFERED=1 && {src}"
	f'ray job submit --address="http://127.0.0.1:8265" '
	f"--runtime-env-json='{json.dumps(runtime_env)}' "
	f"-- python3 {train_script} {model_args} {train_args}"
	)
	cmds.append(
	f"export no_proxy=127.0.0.1 && export PYTHONUNBUFFERED=1 && {src}"
	f'ray job submit --address="http://127.0.0.1:8265" '
	f"--runtime-env-json={shlex.quote(json.dumps(runtime_env))} "
	f"-- python3 {train_script} {model_args} {train_args}"
	)

[gemini-code-assist]

There is both --kl-loss-coef 0.0 and --kl-coef 0.00 specified in grpo_args. Please verify if --kl-coef is a typo or redundant, as --kl-loss-coef is typically used (as seen in test_qwen3_30B_A3B_npu.py). If it is unrecognized by the training script, it might cause the test to fail.

    grpo_args = (
        "--advantage-estimator grpo "
        "--kl-loss-coef 0.0 "
        "--kl-loss-type low_var_kl "
        "--entropy-coef 0.0 "
        "--eps-clip 0.2 "
        "--eps-clip-high 0.28 "
    )