CI: add FFM bringup workflow by gyohuangxin · Pull Request #3596 · ROCm/aiter

gyohuangxin · 2026-06-08T06:10:28Z

Summary

Add a manual FFM bringup workflow that resolves an Artifactory FFM package, builds the base and FFM Docker images, and runs the aiter GEMM smoke test inside the FFM runtime.
Keep image publishing opt-in via push_image so test runs do not overwrite rocm/ffm-dev:latest by default.
Include the Dockerfiles and helper scripts needed to mirror the Jenkins FFM build path in GitHub Actions.

Test plan

Parsed the workflow YAML locally.
Did not run the full workflow locally because it requires AMD internal Artifactory access, private GitHub credentials, Docker BuildKit, and a GPU self-hosted runner with /dev/kfd and /dev/dri.

Add a manual GitHub Actions path to build the FFM image, run the aiter GEMM smoke test, and optionally publish the validated image for bringup testing.

github-actions · 2026-06-08T06:10:39Z

🏷️ CI Guide

Runs automatically on every PR:

✅ Pre-checks (submodule verification, code formatting)
✅ Aiter op tests (gfx942 + gfx950)
✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
`ci:triton-300x`	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
`ci:sglang`	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
`ci:atom`	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
`ci:atom_full`	ATOM accuracy suite for PR and main models from ATOM `models_accuracy.json`
`ci:vllm`	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
`ci:all`	All standard extended tests (excludes `ci:atom_full`)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 3596 --add-label <label>

Copilot

Pull request overview

This PR adds a manually triggered GitHub Actions workflow to bring up an FFM-based runtime, build the required Docker images (base + FFM), run an aiter GEMM smoke/unit test inside that runtime, and optionally push the resulting FFM image tags to Docker Hub.

Changes:

Added a workflow_dispatch “FFM Bringup” workflow that resolves an internal Artifactory FFM package, builds images, runs aiter tests, and optionally pushes rocm/ffm-dev tags.
Added Dockerfiles for a minimal Ubuntu 24.04 base image and an FFM image that pulls FFM artifacts and builds Triton/aiter.
Added helper scripts for FFM environment setup and optional LLVM+Triton build.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
.github/workflows/ffm-bringup.yaml	Manual bringup workflow to build/test (and optionally push) FFM-based images.
.github/ffm/Dockerfile.base	Base Ubuntu image with Python, ROCm SDK deps, and build/test tooling.
.github/ffm/Dockerfile.ffm	FFM image build that downloads FFM package and builds Triton + aiter.
.github/ffm/ffmlite_env.sh	Runtime environment setup script sourced inside the container.
.github/ffm/build_llvm_triton.sh	Helper script to build custom LLVM and then build Triton against it.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+#!/bin/bash
+pkgroot="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+export ROCM_PATH=$(pip show torch | grep ^Location: | cut -d' ' -f2-)/_rocm_sdk_core


+      - name: Push FFM image
+        if: ${{ inputs.push_image }}
+        env:


+    mkdir -p /opt/rocm && \
+    ln -sf ${ROCM_PATH}/lib /opt/rocm/lib
+
+RUN pip install --no-cache-dir --upgrade hip-python -i https://test.pypi.org/simple/


github-actions · 2026-06-08T06:42:54Z

🏷️ CI Guide

Runs automatically on every PR:

✅ Pre-checks (submodule verification, code formatting)
✅ Aiter op tests (gfx942 + gfx950)
✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
`ci:triton-300x`	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
`ci:sglang`	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
`ci:atom`	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
`ci:atom_full`	ATOM accuracy suite for PR and main models from ATOM `models_accuracy.json`
`ci:vllm`	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
`ci:all`	All standard extended tests (excludes `ci:atom_full`)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 3596 --add-label <label>

Run Ruff directly with GitHub annotation output so the pre-check no longer depends on downloading the latest reviewdog release at runtime.

This reverts commit bc5d9a5.

Allow the FFM bringup workflow to run from this PR by adding a path-scoped pull_request trigger and PR-safe defaults for the Jenkins parity test case.

Copy the current aiter checkout into the FFM image instead of cloning ROCm/aiter during docker build, removing the need for the ROCm GitHub SSH key.

Clone AMD-Lightning-Internal LLVM and AMD-Triton over HTTPS with GitHub App installation tokens passed as BuildKit secrets instead of requiring an AMD SSH private key.

Use the aiter-1gpu-runner label as the default FFM bringup runner while keeping the workflow_dispatch input override.

CI: add FFM bringup workflow

406669e

Add a manual GitHub Actions path to build the FFM image, run the aiter GEMM smoke test, and optionally publish the validated image for bringup testing.

gyohuangxin requested review from a team and Copilot June 8, 2026 06:10

Copilot started reviewing on behalf of gyohuangxin June 8, 2026 06:10 View session

Copilot AI reviewed Jun 8, 2026

View reviewed changes

gyohuangxin closed this Jun 8, 2026

gyohuangxin reopened this Jun 8, 2026

gyohuangxin added 2 commits June 8, 2026 14:52

CI: remove reviewdog dependency from Ruff check

bc5d9a5

Run Ruff directly with GitHub annotation output so the pre-check no longer depends on downloading the latest reviewdog release at runtime.

Revert "CI: remove reviewdog dependency from Ruff check"

05cb071

This reverts commit bc5d9a5.

gyohuangxin marked this pull request as draft June 8, 2026 07:00

gyohuangxin added 4 commits June 8, 2026 15:04

CI: trigger FFM bringup on PR changes

d78cd6f

Allow the FFM bringup workflow to run from this PR by adding a path-scoped pull_request trigger and PR-safe defaults for the Jenkins parity test case.

CI: use checked out aiter source in FFM build

4d0bdb0

Copy the current aiter checkout into the FFM image instead of cloning ROCm/aiter during docker build, removing the need for the ROCm GitHub SSH key.

CI: use GitHub App tokens for FFM AMD repos

ab86c0a

Clone AMD-Lightning-Internal LLVM and AMD-Triton over HTTPS with GitHub App installation tokens passed as BuildKit secrets instead of requiring an AMD SSH private key.

CI: run FFM bringup on generic aiter runner

f5b222c

Use the aiter-1gpu-runner label as the default FFM bringup runner while keeping the workflow_dispatch input override.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: add FFM bringup workflow#3596

CI: add FFM bringup workflow#3596
gyohuangxin wants to merge 7 commits into
mainfrom
ci/test-ffm-bringup-gha

gyohuangxin commented Jun 8, 2026

Uh oh!

github-actions Bot commented Jun 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Jun 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

gyohuangxin commented Jun 8, 2026

Summary

Test plan

Uh oh!

github-actions Bot commented Jun 8, 2026

🏷️ CI Guide

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

github-actions Bot commented Jun 8, 2026

🏷️ CI Guide

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants