ggml-cuda : reuse routing IDs across MoE siblings

[GenerelSchwerz]

Overview

Reuse routing IDs on the host across recognized sibling MoE matrix calls. The state is thread-local and bounded to one FFN consumer group, so later ubatches and graph executions cannot inherit a completed entry.

This draft is stacked on draft #41. Commit 0c5347236 is the routing-ID change; after #41 lands, this PR will contain only that commit.

Additional information

The implementation classifies generic gate, up, fused gate_up, and down tensor names. It supports gate/up in either order followed by down, fused gate_up followed by down, and ungated up followed by down. Reuse requires an identified layer and an exact device, data pointer, byte size, four-dimensional shape, and stride match. A mismatch, repeated kind, unknown kind, unknown layer, or down completion clears the pending state and falls back to a fresh device-to-host copy.

ABBA prompt throughput against d39804ac1, with identical -ngl all -fit off placement:

• 188 slots at 8K: 4312.57 -> 4547.52 tokens/s (+5.45%).
• 184 slots at 32K: 4577.81 -> 4831.31 tokens/s (+5.54%).
• 48 slots at 8K: 2062.42 -> 2100.39 tokens/s (+1.84%).
• 48 slots at 32K: 2204.81 -> 2264.91 tokens/s (+2.73%).
• Peak VRAM was unchanged: 15392, 15370, 6132, and 6350 MiB respectively.

At the high-slot points, prefill IDs synchronization fell from 2280 to 760 at 8K and from 7920 to 2640 at 32K. The low-slot fallback path also passed with 1914/2280 staged operations at 8K and 7305/7920 at 32K. Existing staged-path telemetry omits its IDs counters, so the reported low-slot synchronization counts cover only non-staged operations.

Correctness and lifecycle checks:

• Frozen PPL against d39804ac1: 1.0020 +/- 0.00013 on both builds.
• Varied PPL against d39804ac1: 2.2860 +/- 0.10226 on both builds.
• The 1-token warm request plus 128-token deterministic response is byte-identical to d39804ac1 at 8K and 32K. SHA-256 hashes are 28cff857cb93fc95031a77dbb01c6181916f3e23e45ff790836c42de132c79b7 and 20ddc55840791153b6f91d2ccf3b311e6651789efaf0c4387746dd0b7dda0c9d.
• Accounting is unchanged: 7997/31997 cached tokens, 4 evaluated tokens, and 128 returned token IDs.
• The candidate reports CUDA_MoE_Cached, is_cached=1, with nonzero misses and evictions at both contexts.
• Untouched dbeb350d1 PPL is 1.0018 +/- 0.00011 frozen and 2.2851 +/- 0.10242 varied. The candidate changes are 1.17 and 0.006 combined standard errors respectively. Its deterministic difference from dbeb is inherited from ggml-cuda : enable MMQ for cached MoE dispatch #41; this commit is byte-identical to ggml-cuda : enable MMQ for cached MoE dispatch #41 at both contexts.
• test-moe-cache passed under the shared GPU lock, and all server lifecycles exited cleanly.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - Codex implemented and validated the contributor-owned state-machine design under the private-fork owner authorization. The repository owner remains responsible for the submitted change.