Skip to content

perf(ds4): overlap attention across heterogeneous GPUs - #586

Closed
davide221 wants to merge 2 commits into
codex/generic-heterogeneous-stage-plannerfrom
codex/ds4-attention-full-tp-proven
Closed

perf(ds4): overlap attention across heterogeneous GPUs#586
davide221 wants to merge 2 commits into
codex/generic-heterogeneous-stage-plannerfrom
codex/ds4-attention-full-tp-proven

Conversation

@davide221

@davide221 davide221 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • adds an opt-in 25% attention-head split for ratio-4 verifier layers
  • overlaps the R9700 and Strix branches with a packed destination-stream fork and one deferred residual join
  • synchronizes both backends before heterogeneous scheduler teardown
  • adds request-windowed rocprof analysis and records the corrected kernel-level findings

Qualification

  • same-binary control: 88.538 tok/s
  • final two-warmup, seven-run split median: 88.472 tok/s, range 88.309–88.611
  • exact expected response hash: 7/7
  • measured R9700 overlap: 21.20% to 30.67%

The attention split is throughput-neutral and remains opt-in. Kernel traces show that the peer branch adds more work than the main branch saves, so this PR does not claim a decode speedup. The profiler report also records rejected peer-cache and expert-reuse experiments so they are not repeated.

Validation

  • HIP grouped-expert correctness suite passes
  • Python profiler tools compile and expose valid CLI help
  • git diff --check passes
  • restored qualification server checksum matches the proven baseline binary

Stacked on draft PR #577.

Review in cubic

@davide221 davide221 closed this Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant