Repository navigation
Conversation
Co-authored-by: OpenAI Codex <codex@openai.com>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
|
Superseded by local-inference-lab#229. This PR was correctly identified as stacked on the DS4 compressed-MLA contract branch rather than clean GG. The verifier-capacity commit has now been added directly to vllm-project#229, where it shares the same runtime contract and can be reviewed/tested without hidden ancestry. vllm-project#229 documents the K5/K7 MNS64 row envelopes and the SM120 384/512/768-row validation. |
Summary
This is the vLLM half of the fix for compressed-MLA DSpark serving above r27's
previously qualified C24 envelope for both K5 and K7. It gives B12X the
scheduler-owned verifier-row capacity needed for clean C64/C128 planning
instead of leaving the backend at its legacy 256-row assumption. At MNS64 the
derived capacity is 384 rows for K5 and 512 rows for K7.
max_num_seqs * (1 + draft_tokens), capped by the scheduler token budgetDepends on local-inference-lab/b12x#125.
This is a one-commit follow-up to
local-inference-lab/vllm#229,
not a duplicate: that PR reserves the workspace envelope, while this change
supplies the scheduler-owned verifier-row capacity consumed by B12X. The base
is intentionally commit
672aec40dc664de014bd1e92d9a2885d7658c37f, the exacthead of local-inference-lab#229 pinned by the r27 release manifest.
Testing
uvx ruff check ...anduvx ruff format --check ...: pass for all four changed files/opt/venv/bin/python -m pytest -q tests/models/deepseek_v4/test_b12x_compressed_mla_workspace.py: 26 passed/opt/venv/bin/python -m pytest -q tests/kernels/test_compressor_kv_cache.py -k save_partial_states_writes_every_verifier_row: 2 passedThe two production files are byte-identical to the qualified image. No reasoning, sampling, model, draft-depth, or kernel changes are included; K5 remains the release default unless the separate K7 qualification succeeds.
AI assistance was used for diagnosis, implementation, and test preparation; the submitter reviewed the diff and ran the listed tests and model evaluation.