[AIROCMLIR-49] LLVM Upstream Merge - April 2026

[mirza-halilcevic]

Motivation

Pulled in ~6 months of upstream changes, up to commit eaf55a972c2f174bc99390f8f6701ae816645100.

Technical Details

AMDGPU

Structural Reorganization

• Split from single AMDGPU.td into: AMDGPUBase.td, AMDGPUOps.td, AMDGPUAttrs.td, AMDGPUEnums.td, AMDGPUTypes.td.

Op Rename

• amdgpu.scaled_ext_packed816 → amdgpu.scaled_ext_packed_matrix
  • firstScaleLane: IntIsOneOf<[0, 1]> → IntIsOneOf<[0, 16]>
  • firstScaleByte: IntMaxValue<2> → IntMaxValue<3>

New Operations

• amdgpu.sparse_mfma — Sparse MFMA (smfmac) on gfx942+. Operands: sourceA, sourceB, destC, sparseIdx. Attrs: m, n, k, cbsz, abid.
• amdgpu.scaled_wmma — Scaled WMMA on gfx1250+. Operands: sourceA, sourceB, destC, scaleA, scaleB. Shapes: 16×16×128, 32×16×128.
• In-LDS Barrier Ops (gfx1250+): ds_barrier_init, ds_barrier_poll_state, ds_async_barrier_arrive, ds_barrier_arrive, ds_barrier_state_phase, ds_barrier_state_pending_count, ds_barrier_state_init_count, ds_barrier_state_phase_parity.
• TDM Ops (gfx1250+): make_dma_base_gather, make_dma_base_scatter — descriptor-building for gather/scatter.

New Types

• TDMGatherBaseType, TDMScatterBaseType for TDM descriptor results.

ROCDL

New Operations

• Cluster ops (gfx1250+): cluster.id.x/y/z, wave.id.
• Control flow: s.nop with count attribute.
• Named barriers (gfx1250+): s.get.named.barrier.state, s.wakeup.barrier.
• Load-transpose (gfx1250+): Global (global.load.tr4.b64, global.load.tr8.b64, global.load.tr6.b96, global.load.tr8.b128) and DS variants.
• Cluster load-to-LDS (gfx1250+): cluster.load.async.to.lds.b8/b32/b64/b128.
• Prefetch (gfx1250+): global.prefetch, flat.prefetch.
• Atomic barriers (gfx1250+): ds.atomic.barrier.arrive.rtn.b64, ds.atomic.async.barrier.arrive.b64.
• Math intrinsics: rocdl.tanh, sin, cos, rcp, exp, exp2, log, sqrt, rsq.

MFMA/WMMA Rework

• MFMA ops now use ROCDL_Mfma_IntrOp with explicit ABType, CDType (no more variadic $args).
• New ROCDL_Mfma_Scale_IntrOp for scaled MFMA, ROCDL_Smfmac_IntrOp for sparse MFMA.
• WMMA uses new classes (ROCDL_WMMA_IntrOp, ROCDL_WMMA_Opsel_IntrOp, etc.) with explicit operands/attrs.
• New scaled WMMA: wmma.scale.f32.16x16x128.f8f6f4, wmma.scale16.f32.16x16x128.f8f6f4.
• New smfmac variants for 16×16×64, 16×16×128, 32×32×32, 32×32×64.

Barrier API Changes

• BarrierInitOp: $id → $memberCnt.
• ROCDL.barrier deprecation notice; prefer gpu.barrier.

New Type

• ROCDLGlobalBuffer: LLVM_PointerInAddressSpace<1>.

AMDGPUToROCDL / GPUToROCDL

New Lowerings

• Sparse MFMA: SparseMFMAOpLowering for amdgpu.sparse_mfma → ROCDL smfmac (gfx942+).
• Scaled WMMA: ScaledWMMAOpLowering for amdgpu.scaled_wmma (gfx1250+).
• Scaled Ext Packed Matrix: ScaledExtPackedMatrixOpLowering (gfx1250+).
• In-LDS barriers: Lowerings for all ds_barrier_* ops.
• TDM: Lowerings for make_dma_base_gather/scatter on gfx1250+.
• MemoryCounterWaitOp: New tensor counter and WaitTensorcntOp.

API Changes

• convertMFMAVectorOperand → packSmallFloatVectorOperand.
• castMFMAScaleOperand → castScaleOperand (supports vector<8xi8> → i64).
• MFMA attrs (cbsz, abid, blgp) moved from operands to attributes.
• BF16 WMMA: on gfx1250 stays as BF16; on older chips bitcast to i16.
• DPP perm argument handling tightened (assumed non-null for known modes).

gpu.barrier Lowering

• Simple Pat<GPU_BarrierOp, ROCDL_BarrierOp> replaced by GPUBarrierOpLowering that handles memfence address spaces and chipset-specific behavior.

Shared Infrastructure

• New amdgpu::populateCommonGPUTypeAndAttributeConversions() centralizes GPU→ROCDL type/attribute conversions (used by both NVVM and ROCDL paths).

Buffer Descriptor (gfx1250+)

• Updated makeBufferRsrc for new flag layout.
• getNumRecords: with boundsCheck=false, returns (1<<45)-1.

Upstream Patches

1. Missing type converter for ConvertMemrefStore

Affected test:

• Dialect/MHAL/emulate-narrow-type.mlir

Root cause: The upstream memref::populateMemRefNarrowTypeEmulationPatterns now registers ConvertMemrefStore via patterns.insert<>() without the type converter, while all other patterns use patterns.add<>(typeConverter, ...). This means the ConvertMemrefStore conversion pattern does not have a type converter associated, so operand adaptation may not work correctly.

The error is: failed to legalize operation 'memref.store' that was explicitly marked illegal for memref.store on memref<8xi4>.

Fix: Fix is to add the ConvertMemrefStore pattern with the type converter associated.

Diff Files

• diff -rup llvm-project/llvm rocMLIR/external/llvm-project/llvm
  llvm-diff.txt
• diff -rup llvm-project/mlir rocMLIR/external/llvm-project/mlir
  mlir-diff.txt

Test Plan

External Tests

• check-llvm
• check-mlir

CI

• PR
• Nightly
• Weekly

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[umangyadav]

@mirza-halilcevic Github is unable to render such large changes.

1. Can you instead attach diff files. Upstream merge december  #2167 check this PR
2. Can you mention commit hash this upstream merge points to ?

[mirza-halilcevic]

@mirza-halilcevic Github is unable to render such large changes.

1. Can you instead attach diff files. Upstream merge december  #2167 check this PR
2. Can you mention commit hash this upstream merge points to ?

The diff files are too large to be uploaded as attachments (in gigabytes). If needed we can generate the diff with:

git diff HEAD:external/llvm-project/llvm cd54dc7e1339ce77fd6ee7b1f683c5f996eee061 -- llvm
git diff HEAD:external/llvm-project/mlir cd54dc7e1339ce77fd6ee7b1f683c5f996eee061 -- mlir

[justinrosner]

The diff files are too large to be uploaded as attachments (in gigabytes). If needed we can generate the diff with:

git diff HEAD:external/llvm-project/llvm cd54dc7e1339ce77fd6ee7b1f683c5f996eee061 -- llvm
git diff HEAD:external/llvm-project/mlir cd54dc7e1339ce77fd6ee7b1f683c5f996eee061 -- mlir

The diff files have historically only been a couple of hundred lines for both llvm and mlir. If we are seeing the file size be a couple of gb then it seems like something went wrong.

[justinrosner]

Also, when it comes time to merge in this PR can we try to have a clean git history as much as possible? I think having a single commit for bumping the LLVM version, then another commit for any external changes that we need to apply, and then finally a third commit for any rocMLIR changes that were needed because of the upstream bump.

[umangyadav]

The diff files are too large to be uploaded as attachments (in gigabytes). If needed we can generate the diff with:

I meant git diff between upstream SHA and with this branch and not against the develop branch

[mirza-halilcevic]

@umangyadav @justinrosner diff files can be found in the PR description

[umangyadav]

Github is unable to render changes for me. I can not see "approval" button either. I've gone through diff files and changes look good to me. Consider this PR approved from me.

@justinrosner @pabloantoniom can you guys please check changes related to backend/AsyncLoadToLDS diffs etc. ?

[justinrosner]

Github is unable to render changes for me. I can not see "approval" button either. I've gone through diff files and changes look good to me. Consider this PR approved from me.

@justinrosner @pabloantoniom can you guys please check changes related to backend/AsyncLoadToLDS diffs etc. ?

I'm also unable to see the approve button... is this a known problem because so many files have been updated? I don't recall this happening on past upstream merges

Update: I refreshed a few times and it worked for me.

[umangyadav]

One minor comment reported by AI Agent:

`external/llvm-project/mlir/lib/Conversion/GPUToROCDL/LowerGpuOpsToROCDLOps.cpp:725` — target.addIllegalDialect<gpu::GPUDialect, vector::VectorDialect>(); is called right after an existing target.addIllegalDialect<gpu::GPUDialect>();. The second call redundantly re-adds gpu::GPUDialect as illegal; it's not wrong, but the two calls could be consolidated into a single target.addIllegalDialect<gpu::GPUDialect, vector::VectorDialect>();.