DeepSeek V2/V3 with `-mla` option - #12725

Closed

jukofyork wants to merge 24 commits into

ggml-org:masterfrom

jukofyork:mainline-llama-cpp-master--mla

Commits on Apr 2, 2025

Initial commit with all but the MLA graph code done
jukofyork
committed
b4c169f
Fixes
jukofyork
committed
10207b4
Just make `uint32_t n_embd_k` and `uint32_t n_embd_v`
jukofyork
committed
ea3c05b
First working version
jukofyork
committed
1f604a7
Fixed return bug in `DeepseekV2Model`
jukofyork
committed
1de077b
Minor fixes
jukofyork
committed
7f92e7b
More fixes
jukofyork
committed
319e3ef
Renamed `wv_b` to `wv_decompress` to avoid confusion with `_b` biases
jukofyork
committed
ee4b389
Better `_compressed` variable names
jukofyork
committed
c00cd9e
Minor comment and variable name fixes
jukofyork
committed
55ad3a7
Moved `build_attn_mla` to better location
jukofyork
committed
0c86f56
Removed `gguf.MODEL_TENSOR.ATTN_K_B` from `prepare_tensors()` for now
jukofyork
committed
b0c8a43
Bumped `wkv_b` and `wk_b` to use F32.
jukofyork
committed
8c329bc
Use `ggml_mul_mat_set_prec` `GGML_PREC_F32` by default for now
jukofyork
committed
68302ee
Better/shorter variable names and more tidying up of code
jukofyork
committed
937a48d
Fixed `kv_cmpr_pe` name
jukofyork
committed
1fd0aab
Added `n_embd_head_k` as constant
jukofyork
committed
4fb439f

Commits on Apr 3, 2025

Commits on Apr 4, 2025

Commits on Apr 5, 2025

Merge branch 'ggml-org:master' into mainline-llama-cpp-master--mla
jukofyork
authored
f0d514a