Eval bug: tensor parallelism and kv offload crashes

[Piggidragon]

Name and Version

version: 0.3.0-dev (build 10636, commit 4d19b28)
built with GNU 16.2.1 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

i5 13400f + RTX 4070 + RTX 3060

Models

Gemma-4-26b-A4b
Qwen-3.8-27b

Problem description & steps to reproduce

When running both models using tensor parallelism for both GPU's and offloading the kv cache to cpu crashes.
This is an entirely AI generated fix I made that can be used as reference (at least the PR describes the issue in more detail than I understand): GenerelSchwerz#48

Steps to reproduce:

Two CUDA devices, any model with n_head_kv > 1. llama-cli is conversation-only now,
so use llama-completion.

cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="86;89"
cmake --build build -j --target llama-completion

build/bin/llama-completion -m <model>.gguf \
    -ngl 99 -c 4096 -n 32 --temp 0 --seed 1 --no-warmup --jinja \
    -p "Say hello." \
    -sm tensor -ts 55,45 -nkvo

Dropping -nkvo, or using -sm layer, works.

First Bad Commit

No response

Relevant log output

Logs

Gemma:
ggml/src/ggml-cuda/fattn.cu:371: GGML_ASSERT(Q->ne[2] % K->ne[2] == 0) failed

Qwen:
ggml/src/ggml-backend-meta.cpp:826: GGML_ASSERT(src_ss[5].axis == GGML_BACKEND_SPLIT_AXIS_2 || src_ss[5].axis == GGML_BACKEND_SPLIT_AXIS_1 || src_ss[5].axis == GGML_BACKEND_SPLIT_AXIS_0) failed

log1.log

log2.log