Skip to content

Add MI300X FlyDSL BF16 GEMM tasks with strict cold-cache validation - #136

Merged
irvineoy merged 4 commits into
AMD-AGI:mainfrom
zili-amd:aiter-mi300-gemm-tasks
Oct 27, 2026
Merged

irvineoy merged 4 commits into
AMD-AGI:mainfrom
zili-amd:aiter-mi300-gemm-tasks

Conversation

@zili-amd

@zili-amd zili-amd commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add gfx942 (MI300X) variants of three BF16 GEMM tasks under tasks/Aiter-task-mi300: gemm_a16w16_nt_n128_k6144, gemm_a16w16_nt_n2048_k2048, and gemm_a16w16_nt_n6144_k6144.

Each task starts from a working FlyDSL kernel based on AITER dbd8bf5bd624120197a7a26780a8c72201824f0f, with its helper modules joined into one task-local file. The agent optimizes that implementation. The original 13 cases, inputs, reference, comparator and candidate numerical gate remain unchanged.

The baseline is the installed AITER dispatch: hipBLASLt on gfx942, plus AITER's skinny kernel for some small-M cases. All three tasks require baseline correctness; gfx950 diagnostic exceptions do not apply.

Both roles now use cuda_graph_cold_cache: prepare inputs, perform a read-only 1 GiB eviction sweep, then time one complete operator graph replay with same-stream GPU Events. The 20 warmups and 100 samples remain unchanged. Raw samples and cache identity are retained, and the existing current-weight, input-mutation and unseen-draw checks remain active. Graph capture failures cannot silently use Event fallback.

Shared helper, parser, pairing and validator support enforce that timing identity and require graph replay review. The runnable cold-cache reference and focused regression tests are included.

Validation

The example validator config pins the tested runtime image by digest:

docker.io/primussafe/sglang@sha256:1bdb3a9f1c068255d5d631ba4ce80df7f27223f582b8f70036040ea6fc497370

Validation of frozen revision d74c10b9ef1e833082c75cb0fb2f17a8274f0b43 on 2026-10-27:

  • Full Linux x86-64 CPU suite in Docker: 18,860 passed, 155 skipped, 18 subtests passed. Helper audit passed for 441 entrypoints; repository-wide compileall passed.
  • MI300X (gfx942), Python 3.10.12, PyTorch 2.9.1 / ROCm 7.2, FlyDSL 0.3.2: all 21 task actions passed. Both roles passed all 13 cases per task, including current-weight and timed-replay output checks. All baseline numerical gates passed without diagnostic waivers.
  • All 3 task-validator reports are framework-finalized overall_status: PASS, with matching completion markers; all 12 checks passed for each task. GPU actions ran in the pinned runtime. Semantic review used a hash-verified source/evidence mirror, and the original scoring runtime finalized the unchanged GPU evidence with fresh request IDs and evidence digests.
  • Both roles retain 100 raw samples per case (7,800 total), 20 warmups, a 1 GiB read-only eviction sweep outside timing, and exactly one operator graph replay per sample.
  • Real-GPU graph smoke checks: 4 passed. Hosted unit/helper audit and documentation checks passed.

This qualification establishes task correctness and benchmark integrity. It does not establish an optimization speedup.

make docker-run CONFIG=example_configs/task_validator_aiter_mi300x.yaml

The initial GPU results from 2026-10-22 used the earlier warm-graph protocol. They remain in the task READMEs as historical evidence and do not qualify this revision or establish cold-cache speedups. Changing the cache policy changes benchmark identity; both baseline and candidate must be measured again.

zili-amd and others added 2 commits October 22, 2026 16:32
…/Aiter-task-mi300

Each variant is its tasks/Aiter-task original with the gfx942 platform, an implemented starting candidate
(AITER dbd8bf5's FlyDSL split-K HGEMM joined into kernel.py) and an initial-state check in evaluate.py
that follows config.yaml. Workloads, inputs, reference, comparator, timing and baseline entry are the
originals'. example_configs/task_validator_aiter_mi300x.yaml pins the image the variants were run in.

Co-authored-by: Cursor <cursoragent@cursor.com>
…the READMEs

The initial candidates measured 0.23, 0.55 and 0.48 of the baseline under the current task harness, and the
baseline dispatch was unchanged: hipBLASLt for every case, plus AITER's skinny kernel for three small-M cases
of n128_k6144.

Co-authored-by: Cursor <cursoragent@cursor.com>
@zili-amd

Copy link
Copy Markdown
Contributor Author

@irvineoy This one is ready for review too. It does not depend on #134. With #134's harness guard applied, the three starting kernels pass unchanged, and an added globals() write is refused.

Require correct gfx942 baselines and measure one complete operator graph replay
after a read-only 1 GiB eviction sweep. Preserve all workload, numerical, input
mutation and changed-weight controls, retain raw samples and cache identity, and
require graph replay review for both cold graph methods. Add regression coverage
and a runnable timing reference; keep historical warm-cache results distinct.
@irvineoy irvineoy changed the title Add gfx942 (MI300X) variants of three Aiter FlyDSL BF16 GEMM tasks Add MI300X FlyDSL BF16 GEMM tasks with strict cold-cache validation Oct 27, 2026
@irvineoy
irvineoy merged commit 443ee96 into AMD-AGI:main Oct 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants