[Bugfix][Core] Backport vllm-project/vllm#44082: cache the EAGLE/MTP lookahead block in the SWA prefix-cache mask

[Peuqui]

Purpose

Fixes #205.

With a sliding-window KV cache group whose block size is smaller than the
alignment (DeepSeek V4: block 64, window 128, lcm 256) and EAGLE, MTP or
DSpark speculation, the prefix cache never hits. The engine logs
Prefix cache hit rate: 0.0% on every interval and nothing else; every
follow-up turn of a conversation recomputes its whole context.

The cause and the fix are upstream's: vllm-project/vllm#44082 by ivanium,
merged 2026-06-02. The write-side mask registers only the last blocks of each
aligned segment, while the lookup under speculation needs one more
contiguous block, the one past the aligned boundary, which it then drops.
The two sets never meet. @kezboard233 reported this for this repository in
#205, with a root-cause analysis and a hand-merged patch against an older
main; this PR is the same backport against current main, and all credit
for the analysis is theirs. Our four hand-placed hunks came out identical to
the ones listed in #205 section 5, which we take as a good sign for both.

Applying gh pr diff 44082 -R vllm-project/vllm to b711d53: three files
apply cleanly, four hunks are rejected and were placed by hand.

1. single_type_kv_cache_manager.py: self.use_eagle = False in __init__.
   Rejected only because this base has take_pending_boundary_state_offloads
   right after __init__.
2. kv_cache_coordinator.py: from typing import NamedTuple.
3. verify_and_split_kv_cache_groups in upstream's SpecGroup form, keeping
   what this base adds: the prefix_cacheable skip, the "at least one
   cacheable group" assertion, and the lcm_block_size computation.
   alignment_tokens keeps self.lcm_block_size where upstream passes
   self.scheduler_block_size, which this base does not have.
4. The hunk that spans two methods. In find_longest_cache_hit the call site
   takes drop_eagle_block, the per-round flag; left as use_eagle it would
   pass the group-level flag and lose the convergence guard, silently. In
   HybridKVCacheCoordinator.cache_blocks an EAGLE group may cache one block
   past the aligned boundary. We made exactly the mistake [Bug]: DSpark/MTP + 前缀缓存 命中率恒为 0 且静默失效 —— 上游 #44082 可直接 backport 修复 #205 warns about
   in our own first attempt: a mask-only fix that gets hits but never
   registers that lookahead block.

Beyond upstream's diff, this base has its own find_longest_cache_hit
variants (the two no-op managers and the Mamba align manager) that still
named the parameter use_eagle in their bodies; they take
drop_eagle_block now, otherwise they raise NameError. Upstream's three
regression tests build the manager through a helper that passes
scheduler_block_size; here they call KVCacheManager(...) directly, this
base derives the lcm itself. test_mamba_honors_eagle_cache_drop passes the
renamed keyword.

Test Plan

pytest tests/v1/core/test_prefix_caching.py \
       tests/v1/core/test_single_type_kv_cache_manager.py -q
pre-commit run --files <5 files>
pre-commit run mypy-3.10 --hook-stage manual --files <5 files>

Test Result

With this change: 77 passed.

Upstream's three regression tests against unmodified main (b711d53), test
file only:

FAILED test_eagle_swa_alignment_caches_extra_block
FAILED test_eagle_swa_boundary_caches_post_boundary_block
FAILED test_eagle_grouped_swa_siblings_use_same_cache_mask
AssertionError: EAGLE + SWA with sliding_window <= alignment failed to find
any cache hit; the +1 block past each segment boundary must be cached.
3 failed, 62 deselected

pre-commit: all hooks passed; mypy-3.10 manual stage passed.

Serving, which #205 says it could not measure. DeepSeek-V4-Flash NVFP4 +
DSpark (num_speculative_tokens=5), --enable-prefix-caching, pipeline
parallel over two RTX 8000 and three V100, on a fork of this repository that
carries the same backport:

• The same 5131-token request sent twice with the model loaded: 18.5 s, then
  1.5 s, with cached_tokens: 4864 reported for the second one. 4864 is 19 aligned
  segments of 256; the twentieth is the lookahead block EAGLE drops.
• Follow-up turns at 18k context reach the first token in 1.4 to 1.6 s.
  Before, every turn took 85 to 99 s.
• Prefix cache hit rate in the engine log goes from 0.0% to 56.8% over the
  test session.
• A 30k-token needle test returns 4 of 4 both cold and from the cache, so the
  cached blocks are the right ones.

Not a duplicate

gh issue view 205 -R 1CatAI/1Cat-vLLM --comments        # open, no comments
gh pr list -R 1CatAI/1Cat-vLLM --state all --search "205 in:body"
gh pr list -R 1CatAI/1Cat-vLLM --state all --search "44082"
gh pr list -R 1CatAI/1Cat-vLLM --state open --search "cache_block_mask"
gh pr list -R 1CatAI/1Cat-vLLM --state open --search "prefix cache sliding window"

No PR carries this fix. #603 mentions #205 in passing and touches none of the
five files; #239 matches the last search on wording only.

Overlap to be aware of: #617 (open, Mamba prefix-cache retention, backport of
upstream #45845/#47782) and #598 stacked on it change the same three core
files. #617 does not contain #44082: its diff has no drop_eagle_block, no
SpecGroup, no _contiguous_blocks_for_hit and none of the three regression
tests. It does add a reachable_block_mask classmethod to the Mamba manager,
with a different signature (retention_interval, reachable_boundaries),
and it edits HybridKVCacheCoordinator.cache_blocks. The two changes are
independent in substance but will conflict textually in
single_type_kv_cache_manager.py and kv_cache_coordinator.py. Whichever
lands second needs a rebase; I am happy to do that for this one.

AI assistance

AI assistance (Claude) was used for the hand-merge, the comparison with
upstream and with #205, and this text. I have read every changed line and
ran the tests and the server measurements above on my own hardware.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.