Skip to content

fix(ds4): declare compressed MLA verifier-row capacity - #12

Closed
MadeBy561 wants to merge 1 commit into
voipmonitor:fix/gg-ds4-compressed-mla-contract-20260802from
MadeBy561:fix/ds4-decode-row-capacity-pr229
Closed

MadeBy561 wants to merge 1 commit into
voipmonitor:fix/gg-ds4-compressed-mla-contract-20260802from
MadeBy561:fix/ds4-decode-row-capacity-pr229

Conversation

@MadeBy561

@MadeBy561 MadeBy561 commented Aug 7, 2026 •

Copy link
Copy Markdown

Summary

This is the vLLM half of the fix for compressed-MLA DSpark serving above r27's
previously qualified C24 envelope for both K5 and K7. It gives B12X the
scheduler-owned verifier-row capacity needed for clean C64/C128 planning
instead of leaving the backend at its legacy 256-row assumption. At MNS64 the
derived capacity is 384 rows for K5 and 512 rows for K7.

  • derive DSpark's reachable verifier rows from max_num_seqs * (1 + draft_tokens), capped by the scheduler token budget
  • pass that capacity into B12X compressed-MLA planning and fail closed if live decode exceeds it
  • keep non-DSpark and the existing <=256-row contract unchanged

Depends on local-inference-lab/b12x#125.
This is a one-commit follow-up to
local-inference-lab/vllm#229,
not a duplicate: that PR reserves the workspace envelope, while this change
supplies the scheduler-owned verifier-row capacity consumed by B12X. The base
is intentionally commit 672aec40dc664de014bd1e92d9a2885d7658c37f, the exact
head of local-inference-lab#229 pinned by the r27 release manifest.

Testing

  • uvx ruff check ... and uvx ruff format --check ...: pass for all four changed files
  • /opt/venv/bin/python -m pytest -q tests/models/deepseek_v4/test_b12x_compressed_mla_workspace.py: 26 passed
  • /opt/venv/bin/python -m pytest -q tests/kernels/test_compressor_kv_cache.py -k save_partial_states_writes_every_verifier_row: 2 passed
  • B12X CUDA graph/reference tests at 384 (MNS64/K5), 512 (MNS64/K7), and 768 (MNS128/K5) rows: 3 passed
  • exact r27, TP4, official max, probabilistic K5, MNS64: Estonia C64 64/64, C1 260.4 tok/s, C64 2724.4 tok/s, code emission 406-420 tok/s, prefill 11.2-13.2k tok/s
  • same-image TP4/MNS64/K7 integration smoke: Estonia C64 64/64, C1 250.6 tok/s, code emission up to 499.4 tok/s

The two production files are byte-identical to the qualified image. No reasoning, sampling, model, draft-depth, or kernel changes are included; K5 remains the release default unless the separate K7 qualification succeeds.

AI assistance was used for diagnosis, implementation, and test preparation; the submitter reviewed the diff and ran the listed tests and model evaluation.

Co-authored-by: OpenAI Codex <codex@openai.com>
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

@voipmonitor

Copy link
Copy Markdown
Owner

Superseded by local-inference-lab#229. This PR was correctly identified as stacked on the DS4 compressed-MLA contract branch rather than clean GG. The verifier-capacity commit has now been added directly to vllm-project#229, where it shares the same runtime contract and can be reviewed/tested without hidden ancestry. vllm-project#229 documents the K5/K7 MNS64 row envelopes and the SM120 384/512/768-row validation.

@voipmonitor voipmonitor closed this Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants