feat(streaming): GGUF support in extract pipeline (browse-level)

[mvkorobkov]

Summary

Adds GGUF to the streaming-extract pipeline alongside safetensors. Today GGUF input is routed through the in-memory load_model_dir_validated path which dequantises every tensor to f32 in RAM — fine for small models but architecturally unworkable for ≥70B GGUFs (Kimi K2.6 at 554 GB, DS-V4-Flash at 127 GB).

Design — TensorSource enum

enum TensorSource {
    Safetensors { shards, index },   // existing
    Gguf(GgufTensorSource),          // new
}

StreamingContext::new detects the input format (a .gguf file directly, or a directory whose first / largest .gguf shard is used as the entry point) and constructs the appropriate variant. Each stage now calls self.tensor_source.get_tensor_f32(key) for the canonical 2D dequant path. The MXFP4 raw-pair access (DeepSeek-V4 packed gate_up_proj_blocks / down_proj_blocks) stays safetensors-only — GGUF has no equivalent packed format.

GGUF specifics

• Multi-shard splits are handled via GgufFile::open (feat(gguf): multi-shard reader for *-NNNNN-of-NNNNN.gguf splits #136). Each shard is mmap'd eagerly — virtual address space only, the OS pages in only the tensors we touch.
• Per-tensor read does data_offset + offset into the right shard, slices tensor_data_size bytes, and dequantises via larql_models::quant::ggml::dequantize (Q4_K / Q5_K / Q6_K / Q8_0 / BF16 / F32 etc — all already supported).
• Dim ordering follows load_gguf's reshape-to-(dims[1], dims[0]) convention. Canonical FFN orientation (the in-memory loader's orient_in_place) is applied here too, driven by (hidden_size, intermediate_size) from the detected architecture — without it tensor.shape()[0] would be hidden instead of intermediate for some quants and downstream matmul would produce NaN.

CLI routing

Input	Level / quant	Path
safetensors	any	streaming
GGUF	--level browse (and no quant)	streaming (new)
GGUF	attention / inference / all	in-memory (unchanged)
GGUF	any level with --quant q4k	in-memory (unchanged)

Inference / Q4K levels for GGUF still need the StreamingWeights writer subsystem ported to read tensors via ggml::dequantize per tensor — tracked as a follow-on PR. The stage gate returns a clear VindexError::Parse(…) if the user requests an unsupported level/quant combo for GGUF input.

Validation

• DS-R1-0528-Qwen3-8B-Q3_K_L (10 GB, dense, mixed Q3_K / Q4_K / Q6_K) → 3.4 GB gate_vectors.bin + 1.2 GB embeddings.bin written cleanly through the streaming path. Resident memory stays at the per-layer-tensor scale, not the full-model scale.
• 1074 existing vindex unit tests pass.

Known gap (pre-existing)

The streaming pipeline's MoE branch looks up per-expert 2D keys (mlp.experts.K.gate_proj.weight) which GGUF stores as 3D-packed tensors (blk.L.ffn_gate_exps.weight, [hidden, intermediate, n_experts]). Both streaming and in-memory load_gguf currently skip these 3D tensors. Unpacking them lives in the same follow-on PR as inference-level GGUF.

Test plan

• cargo check --workspace clean
• cargo test -p larql-vindex --lib — 1074 / 1074 pass
• End-to-end on dense GGUF (Qwen3-8B Q3_K_L) — gate_vectors.bin + embeddings.bin populated correctly
• End-to-end on multi-shard GGUF (Kimi K2.6 at browse level) — pending, this PR unblocks it
• Inference / Q4K for GGUF — follow-on PR