#67 llama.cpp emits DeepSeek-V2/V3 (and Kimi K2) MLA geometry in the GGUF metadata under {arch}.attention.* and {arch}.rope.dimension_count. `to_config_json` was dropping every one of these fields, so the parsed ModelConfig had MLA disabled and PR #96's absorption never fired for GGUF-sourced inputs. This surfaces the relevant fields into the HF-shaped config the parser consumes: - `attention.q_lora_rank` → `q_lora_rank` - `attention.kv_lora_rank` → `kv_lora_rank` - `attention.key_length[_mla]` → `qk_nope_head_dim` (= key_length − rope.dim) - `attention.value_length[_mla]`→ `v_head_dim` - `rope.dimension_count` → `qk_rope_head_dim` For per-head dims the loader prefers the `_mla` variants when present — those carry the pre-absorption (DS-V3-standard) split that `mla_absorb::absorb` operates on. Kimi K2.6's GGUF exposes both forms (192/128 for `_mla`, 576/512 absorbed); we want 192/128. Verified against Kimi K2.6 UD-Q8_K_XL GGUF metadata (the unsloth name is misleading — actual tensor types are BF16 + F32 + Q4_0, all already supported by larql's existing dequant). Three new tests cover: 1. Kimi K2.6-shaped metadata → full MLA fields populated, MLA detected 2. Non-`_mla` variant fallback (DS-V2 with key_length only) 3. Non-MLA architectures (llama) keep their fields absent 281/281 larql-models tests pass. Combined with PR #96 + #103 + #133, this unlocks inference-level extraction of Kimi K2 family and any other DeepSeek-V2/V3 GGUF that exposes the standard MLA metadata.

Kimi K2 support #67

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions