Skip to content

feat: support NineToothed RMSNorm on Ascend - #1004

Open
voltjia wants to merge 1 commit into
masterfrom
feat/ninetoothed-ascend-rmsnorm
Open

voltjia wants to merge 1 commit into
masterfrom
feat/ninetoothed-ascend-rmsnorm

Conversation

@voltjia

@voltjia voltjia commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Allow WITH_NINETOOTHED to select the Ascend backend in addition to the existing NVIDIA Triton backend.
  • Pass the NineToothed backend and platform through the InfiniOps code generator.
  • Generate Ascend RMSNorm FP32/FP16 rank-2/3/4 and block-size 256/512 variants.
  • Link the generated standalone Ascend C++ launchers with the Ascend runtime.
  • Register RmsNorm implementation index 9 for Ascend and share its launcher setup with the NVIDIA specialization.
  • Explicitly skip BF16 for Ascend NineToothed RMSNorm because triton-ascend 3.2 currently crashes in triton-adapter-opt for that AOT configuration.

Motivation

InfiniOps already integrates NineToothed-generated RMSNorm on NVIDIA. This extends the same generated-native-kernel path to Ascend 910B4 instead of requiring a Python loader at inference time.

Type of Change

  • feat — new feature / new operator / new platform
  • Breaking change for callers that explicitly request Ascend index 9 with BF16 before that compiler limitation is fixed

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Ascend (WITH_ASCEND)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

# Ascend 910B4, final InfiniOps commit c6e2d25
$ python3.11 -m pytest -q tests/test_rms_norm.py --devices npu -k "npu-9" --tb=short
24 passed, 13 skipped, 37 deselected in 1.72s

# NVIDIA A100, final InfiniOps commit c6e2d25
$ python3 -m pytest -q tests/test_rms_norm.py --devices nvidia -k "cuda-9" --tb=short
37 passed, 37 deselected in 0.51s

# Generator
$ python3.11 -m pytest -q tests/test_generate_ninetoothed_ops.py --tb=short
2 passed in 0.07s

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes WITH_CPU=ON, WITH_NVIDIA=ON, WITH_NINETOOTHED=ON build passed RMSNorm NineToothed index 9: 37 passed
Iluvatar No N/A - not affected N/A
MetaX No N/A - not affected N/A
Cambricon No N/A - not affected N/A
Moore No N/A - not affected N/A
Ascend Yes WITH_CPU=ON, WITH_ASCEND=ON, WITH_NINETOOTHED=ON build passed FP32/FP16: 24 passed; BF16: 13 skipped for the documented triton-ascend adapter crash
Full `pytest` output
Ascend:
24 passed, 13 skipped, 37 deselected in 1.72s

NVIDIA:
37 passed, 37 deselected in 0.51s

Generator:
2 passed in 0.07s

Benchmark / Performance Impact

N/A. This PR adds an Ascend integration path and does not claim a performance result.

Notes for Reviewers

This branch is intended to stack on these upstream changes:

The Ascend validation used NineToothed 9838a1e3, ntops fcc2ade5, and InfiniRT 5175a694 on an idle 910B4. The generated launchers embed kernel binaries, register them through rtDevBinaryRegister/rtFunctionRegister, and invoke them through rtKernelLaunch.

CI Notes

At final head c6e2d25, clang-format, Ruff, documentation build, matrix generation, and the NVIDIA unit job passed. The repository platform matrix also reported Ascend, Cambricon, and MetaX failures.

The latest master run at 5653c00 had already failed the same CI workflow in its Ascend and MetaX jobs. The PR Ascend log fails while building stock AscendC workspace kernels, before NineToothed code generation. The Cambricon errors are native Cambricon embedding/mul type-conversion compile errors and do not reference files changed by this PR. Direct affected-platform builds and tests above pass with the intended upstream dependency stack.

@voltjia
voltjia requested a review from a team October 19, 2026 11:23
@voltjia
voltjia force-pushed the feat/ninetoothed-ascend-rmsnorm branch from b4489dc to c6e2d25 Compare October 19, 2026 11:42

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant