bug: lmCacheConfig with Blackwell 6000 pro and CUDA 13

[dotmobo]

Describe the bug

Hi,

Could you please release a vLLM Docker image with CUDA 13? The current images still use CUDA 12.8, and for compatibility with the latest Blackwell GPU we need CUDA 13.

Thanks,
Morgan

To Reproduce

Deploy gpt-oss-120b under 2 blackwell 6000 pro with lmcache/vllm-openai:v0.4.2 and lmCacheConfig to true :

(EngineCore_DP0 pid=100) ERROR 03-24 14:38:36 [core.py:1102] RuntimeError: Worker failed with error 'CUDA error: no kernel image is available for execution on the device
(EngineCore_DP0 pid=100) ERROR 03-24 14:38:36 [core.py:1102] Search for `cudaErrorNoKernelImageForDevice' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.
(EngineCore_DP0 pid=100) ERROR 03-24 14:38:36 [core.py:1102] CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

Expected behavior

No response

Additional context

No response

[dotmobo]

The error occurs when I enable lmCacheConfig:

lmCacheConfig:
    enabled: true
    cpuOffloadingBufferSize: "64"
    diskOffloadingBufferSize: "0"

If I disable lmCacheConfig, the issue does not occur during inference.
It seems to be linked to the KV cache transfer with lmCache.

[ruizhang0101]

Hi, could you also raise the issue in lmcache repo since they are working on kvcache transfer compatibility. https://github.com/lmcache/lmcache

[rorubyy]

The v0.4.3 stable image uses CUDA 12.6 and does not include sm_120 kernels for Blackwell GPUs. The latest-nightly image has sm_120 support compiled in — switching to it fixes the no kernel image is available for execution on the device error on RTX PRO 6000 Blackwell.

[Tobi-Adesoye]

Running modern Blackwell architectures on early CUDA 13 runtimes frequently introduces silent kernel misalignments, especially around flash-attention structures and hardware-level memory caching layers. When the caching library tries to interface with the new hardware allocation matrices, precision mismatches can occur.

If you are running into stability drops or unexpected behavior here, ensure your custom Triton or CUDA kernels aren't hitting extreme micro-variance underflows when translating activations on the new architecture. I've been working extensively on anchoring tensor variance during normalization to prevent these exact kinds of hardware-edge-case failures on high-compute nodes.