Repository navigation
pd: import dense subprocess KV into hybrid GPU cache groups (#258) - #283
Merged
Merged
Conversation
PagedKV.write_span splits dense model-layer-order KV (subprocess producers) into the receiver's per-geometry groups and validates all shapes before any copy. From fix/258-dense-hybrid-kv (#259). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Subprocess prefill producers export KV as one dense tensor in model-layer order. A hybrid GPU receiver (
PagedKVwith several attention-geometry groups) expected one tensor per group, so dense imports into hybrid models failed (#258).Changes
From
fix/258-dense-hybrid-kv(#259), only:opt_prime/opt_prime/accel/pd/paged.py:write_spansplits a dense model-layer-order span into the receiver's groups by layer index, and validates group count, rank and every group's shape before copying any group. Diffed againstoptimus-rm: the only change is this function.opt_prime/tests/test_pd_dense_hybrid.py(new).Verification
pytest -q tests/test_pd_dense_hybrid.py: 4 passed.optimus-rmversion ofpaged.py: 4 failed.Compatibility and limits
ValueErrorbefore any write instead of failing midway.Related Issue
Closes #258
Supersedes #259.
🤖 Generated with Claude Code