llama : add an attention split separate from the tensor split

[Piggidragon]

Overview

Depends on #66. That PR's commits are included in this branch because the base repository has
no branch to target. Only the last commit, llama : add an attention split separate from the tensor split, belongs to this PR. Merge #66 first and this diff reduces to that commit.

Adds --attn-split (-as), the fraction of the attention heads each GPU gets under
--split-mode tensor.

That share decides two things: how much of a host-resident KV cache a device receives, and how much
attention work it does. Neither has to follow the memory split. On an asymmetric pair the device with
the slow host link is exactly the one that should hold less of the cache, while the memory split still
wants to fill both cards.

Unset, it follows --tensor-split and nothing changes. Only the attention tensors and the cache move;
the rest of the model still follows --tensor-split. A linear-attention layer is a different
mechanism and keeps --tensor-split. The share is rounded to whole heads by the granularity rules
that already keep GQA consistent, so a ratio the head count cannot express lands on the nearest one it
can.

Stacked on the host-cache correctness PR, which is what makes a host-resident cache work under
-sm tensor at all.

This is a new CLI and public API addition, so per CONTRIBUTING it carries a higher bar. The argument
for it over the existing mechanism: --tensor-split cannot express it. With a device-resident cache
both devices run concurrently inside a layer and the collective is a barrier, so --tensor-split
wants the compute ratio; with a host-resident cache the cache placement wants the bandwidth ratio.
One knob cannot serve both, and the measurements below show the second one is worth several times more
than the first on this pair.

Testing

Two GPUs, RTX 4070 (gen4 x16) + RTX 3060 (gen3 x4), CUDA, NCCL, stock clocks.
Qwen3.8-27B-UD-Q5_K_M, a 56935-token prompt from this repository's docs, -c 65536 -n 128 -ngl 99,
q8_0 cache, -sm tensor -ts 50,50 -nkvo --kv-cpu-pinned --recurrent-state-offload. The model has
n_head_kv = 4, so the whole-head shares available are 2,2 / 3,1 / 4,0.

--attn-split	rounds to	pp	tg
(follows -ts)	2,2	433.33	2.47
0.75,0.25	3,1	475.52	3.55
3,1	3,1	475.69	3.56
1,0	4,0	501.97	7.49

3.03x generation from moving the cache off the slow card, and +16% prefill. 0.75,0.25 and an
explicit 3,1 agree to within noise, which is the rounding working: a ratio the head count cannot
express takes the nearest one it can, and it lands strictly between its two neighbours.

Setting --attn-split when it is not used costs nothing. llama-bench -ngl 99 -ts 50/50 -r 3,
all 16 rows against the parent PR without this commit:

model	sm	nkvo	pp512 parent	pp512 this	tg128 parent	tg128 this
Qwen3.8-27B-UD-IQ2_M	layer	0	755.87	752.83	26.38	26.34
	layer	1	532.98	532.78	8.68	8.66
	tensor	0	610.71	609.89	30.96	30.91
	tensor	1	588.78	589.17	25.61	25.64
gemma-4-26B-A4B	layer	0	2732.69	2733.83	103.83	103.90
	layer	1	1941.45	1986.41	42.87	42.98
	tensor	0	1685.83	1688.82	85.84	86.09
	tensor	1	1339.58	1343.70	34.97	35.37

tests/test-llama-archs.cpp runs the tensor-split matrix a third time with all attention heads on the
first device (attn_split = 1,0) and the cache in host memory, checking the logits against CPU.

test-llama-archs -s 1 at 1, 2, 3 and 4 virtual CUDA devices: passes. test-arg-parser: passes.
Built with -DLLAMA_FATAL_WARNINGS=ON.

Not done here

• Deriving the split automatically from the measured bandwidth.
• -as for llama-bench, which only takes -sm, -ts, -ub, -ngl and -fa; the sweeps below had
  to be run through llama-completion.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - implemented by an agent on my instruction, see the Assisted-by: commit trailers.

[GenerelSchwerz]

Automated preliminary review by Codex; the repository owner plans a separate manual review.

Verdict: FAIL.

Blocking

• common/arg.cpp:2944-2955 uses std::stof without checking full-token consumption or validating the result, so negative, NaN, infinite, partially parsed, and inconsistent zero-sum splits are accepted. include/llama.h:332-334 exposes the same unchecked input to C callers, and src/llama-model.cpp:1235-1237 copies it unchanged. It then reaches cumulative division and float-to-int64_t conversion at src/llama-model.cpp:836-866. Validate complete parsing, finiteness, non-negativity, and a positive total at both the CLI/environment parser and the model API boundary.
• PR ggml-meta : split a host-resident KV cache by head #66's reported Gemma4/iSWA result remains 235.0273 with host KV versus 227.2279 with device KV under tensor split, a 3.43% perplexity gap. This PR depends on ggml-meta : split a host-resident KV cache by head #66 and advertises that host-cache configuration; src/llama-model.cpp:831-866 applies attn_split to it. Resolve or explicitly exclude iSWA before merging that advertised use case.

Will slow review

• The permanent CLI/environment surface at common/arg.cpp:2938-2957 and public C API field at include/llama.h:332-334 have no dedicated issue or design approval. Please get agreement on the API surface before merging it.
• common/arg.cpp:2940-2943 promises the nearest whole-head share, but src/llama-model.cpp:838-866 implements rotating cumulative-floor allocation. For example, a 0.4/0.6 split over four unit-granularity heads gives the first device one head, not the nearest two. Make the behavior and documentation agree.
• The newest multi-stream path at ggml/src/ggml-backend-meta.cpp:1468-1483 lacks regression coverage. The added host-KV configurations at tests/test-llama-archs.cpp:1318-1329 reach get_logits, which creates a one-sequence batch at tests/test-llama-archs.cpp:1048-1057; it does not exercise ne[3] > 1. Add a parallel host-KV/custom-split case.

Developmental progress

The broad #57 predecessor was split. This version fixes the maximum-device off-by-one at common/arg.cpp:2948, preserves scheduler-copy source names at ggml/src/ggml-backend.cpp:1037-1048, and adds baseline host-KV and custom-split coverage at tests/test-llama-archs.cpp:1318-1329.

Reviewed head: 1311c7b