Skip to content

fix(rccl): Extend fp8 device arithmetic optimizations and restore unroll 8 in gfx1250 device codegen - #9537

Draft
KawtharShafie wants to merge 2 commits into
rccl/gfx1250from
users/KawtharShafie/rccl-gfx1250-fp8-fix
Draft

fix(rccl): Extend fp8 device arithmetic optimizations and restore unroll 8 in gfx1250 device codegen#9537
KawtharShafie wants to merge 2 commits into
rccl/gfx1250from
users/KawtharShafie/rccl-gfx1250-fp8-fix

Conversation

@KawtharShafie

Copy link
Copy Markdown
Contributor

Restore unroll 8 in gfx1250 device codegen so FP8 kernels link and launch (RCCL_UNROLL_FACTOR=3). Extend gfx950 FP8 device optimizations to gfx1250

Motivation

FP8 ReduceScatter on gfx1250 fails on rccl/gfx1250 after #8880 narrowed device codegen to unroll 16/32 only. FP8 Ring/Simple requires the unroll-8 kernel variant and fails with invalid device function. gfx1250 was also excluded from FP8 device optimizations in rccl_float8.h.

Technical Details

  • generate.py: Build unroll 8, 16, and 32 for gfx1250 (unroll 8 required for FP8 launch and RCCL_UNROLL_FACTOR=3; 32 remains the default per commSetUnrollFactor).
  • rccl_float8.h: Extend gfx950 FP8 cast/upcast optimizations to gfx1250; use gfx942-style hadd intrinsics (gfx950 scalef32 is unsupported on gfx1250).

Related: #88804ee4eb0

Issue Tracking

JIRA ID: ROCM-286794

Test Plan

  • Build RCCL for gfx1250 (./install.sh -l)
  • Run reduce_scatter_perf @ 64M, MPI 4 ranks, Ring/Simple
  • Compare FP32 and FP8 with default unroll and RCCL_UNROLL_FACTOR=3 (unroll 8)
  • Compare against rccl/gfx1250 baseline (c318fa3) and prior good build (e0ebb75)

Test Result

Build FP8 Ring/Simple FP8 busbw (unroll 8)
c318fa3 (baseline) FAIL (invalid device function) ...
This PR PASS ...
e0ebb75 (reference) PASS ...

FP32 Ring/Simple behavior is unchanged from the baseline. FP8 Ring/Simple no longer fails on the default path and completes successfully. With RCCL_UNROLL_FACTOR=3 (unroll 8) and applying the fp8 device optimizations to gfx1250, FP8 bandwidth improves over both the broken baseline and the prior reference build.

Submission Checklist

…link and launch (RCCL_UNROLL_FACTOR=3). Extend gfx950 FP8 device optimizations to gfx1250
@KawtharShafie
KawtharShafie requested a review from mustafabar July 31, 2026 02:54
@mustafabar mustafabar changed the title fix(rccl): Restore unroll 8 in gfx1250 device codegen and extend fp8 device optimizations fix(rccl): Extend fp8 device arithmetic optimizations and restore unroll 8 in gfx1250 device codegen Jul 31, 2026
Clamp default kernel unroll to 8 on gfx1250 when using LL protocol,
avoiding GPU memory faults with unroll 32. Only apply when unroll 8
was built; warn once if the user sets RCCL_UNROLL_FACTOR=5 (unroll 32).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant