Skip to content

Update SM90 FlashKDA benchmark results - #120

Open
icavan wants to merge 2 commits into
mainfrom
icavan/sm90-perf-update
Open

Update SM90 FlashKDA benchmark results#120
icavan wants to merge 2 commits into
mainfrom
icavan/sm90-perf-update

Conversation

@icavan

@icavan icavan commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • remove the two obsolete benchmark Markdown generator scripts
  • replace the legacy CUTLASS C++ SM90 numbers with current CuTe DSL FlashKDA measurements
  • add intracard CP-auto vs CP-off results for long and ragged sequence shapes
  • update README highlights and point reproduction instructions at the benchmark scripts directly

Why

SM90 now defaults to the CuTe DSL FlashKDA backend, but the published Hopper data still described the older C++ implementation. This updates the documentation to reflect the current backend and includes the low-head, long-sequence cases where intracard CP is intended to help.

Performance

Measured on an H200 141GB / SM90 GPU with 132 SMs, PyTorch 2.9.1+cu129, Triton 3.5.1, and FLA v0.5.0:

  • FlashKDA vs FLA: 2.39x average across 28 fixed/variable-length configs; 1.38–7.80x range
  • intracard CP engaged in 28/32 configs: 4.11x geometric mean over serial FlashKDA; 1.93–7.21x range

Validation

  • pre-commit run --all-files
  • python -m pytest tests/test_kda_sm90_prefill_vs_fla.py tests/test_kda_sm90_intracard_cp.py -q
    • 28 passed, 15 skipped
  • python benchmarks/bench_kda_sm90_prefill.py --mode both
  • python benchmarks/bench_kda_sm90_cp.py

icavan added 2 commits August 9, 2026 13:01
Signed-off-by: shuyan.ycf <shuyan.ycf@antgroup.com>
Signed-off-by: shuyan.ycf <shuyan.ycf@antgroup.com>
@icavan
icavan marked this pull request as ready for review August 9, 2026 06:06
@cherhh
cherhh self-requested a review August 9, 2026 06:27

@cherhh cherhh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants