Repository navigation
Upgrade the MI355X default runtime to ROCm 10 - #123
Merged
Merged
Conversation
Pin the new scoring image, adapt SDK loading and task dependencies, and update all six evaluation-tool runtimes with matching documentation and regression coverage. Keep qualification pending for GPU ASan and the existing fused-MoE numerical failures.
This was referenced Oct 16, 2026
Resolve runner and task integration conflicts, bind the MLA regression to its validated source, reject ambiguous Triton ABI metadata, and stabilize process-exit observation in the materialization test. Set the declared FP8 fused-MoE normalized-error limit to 1.2% and check changed expert IDs and routing weights during timed replay.
Keep the original hidden/weight perturbation and add a separate routing draw from restored inputs. Validate each draw, restore all inputs on failure, and retain stale-routing and input-mutation controls.
Use memcpy for both signed-zero representations so the native buffer-integrity regression remains meaningful under fast-math optimizations. Keep the byte-exact rejection, restoration assertions and compiler flags unchanged.
Apply the same limit to baseline and candidate correctness, measured outputs and both replay draws. Keep reference math, inputs, samples and timing intact. Update the documented contract and boundary regressions. Validation: 110 focused CPU regressions, 345 Linux framework tests, and a fresh MI355X task-validator PASS with current-source/report binding.
Keep two pipeline stages, add page-routing correctness checks, and reset measured outputs on every invocation for both roles. Preserve the original reference, six workloads, 1% numerical gate and graph sample counts.
Exclude only the two newly covered MLA check helpers from the historical comparison. Retain the original protected hash and all core assertions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The MI355X default uses ROCm 7.2, while the ROCm 10 image changes Python, Triton, FlyDSL and SDK library layout. This change selects the digest-pinned SGLang 0.5.20 / ROCm 10 image and integrates the runtime changes with current main.
Changes
Validation
Hardware: AMD Instinct MI355X on Linux. Scoring image:
lmsysorg/sglang@sha256:e20849665c105d389ef91d23c0dc73931aaa6f02056dd10e7b43e4f16c79df69.task_validatorreports are PASS for five task packages: batched INT8 GEMM, INT8 preshuffle GEMM, HGEMM, FP8 fused-MoE and MLA. The whole runs and post-processing completed successfully. Captured actions, report markers, 85 raw task files and 90 materialized files were checked against the final source.lmsysorg/sglang@sha256:1438dd8b9000adaefb5828edb1cc5713992815d4296f4ad1a1e98dee4e9c7e5f) reproduced the original two-wave error on 104/120 checks, with the same frozen inputs as 0.5.20. Both images contain the same Triton compiler binary. This is an MLA diagnostic, not full-catalog qualification of the newer image; the default image remains unchanged.Limits
Original failures, complete validation reports, source bindings and diagnostic operands are retained with the experiment records.