From 5f6fdb15ab4ca517703da899de43b8504944eb6e Mon Sep 17 00:00:00 2001
From: Niko Maroulis <niko@hackthebox.eu>
Date: Tue, 15 Sep 2026 18:56:28 -0400
Subject: [PATCH] Bump llama.cpp to 72b590d65 (b10989+1, +46 from b6b003d2c),
 release v0.8.51

No NIF-facing header changed; RPC protocol major goes 6 -> 7 (SET_TENSOR
cache-flag byte), so both ends of a two-node split must rebuild together.
The same upstream change stops the RPC hash cache persisting activations.

Verified on Metal (M4 Max): 430 passed default, 573 passed with
smoke/embeddings/slow/mtp, 6 passed MTPE4BSidecarTest.
---
 CHANGELOG.md                   | 36 ++++++++++++++++++++++++++++++++++
 Makefile                       |  2 +-
 lib/llama_cpp_ex/rpc/server.ex |  2 +-
 mix.exs                        |  2 +-
 vendor/llama.cpp               |  2 +-
 5 files changed, 40 insertions(+), 4 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index be1aa7c..774a048 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,41 @@
 # Changelog
 
+## v0.8.51
+
+llama.cpp bumped to `72b590d65` (one past
+[`b10989`](https://github.com/ggml-org/llama.cpp/releases/tag/b10989),
+upstream v0.4.1), +46 from `b6b003d2c` (b10944). No change to any header the
+NIF includes (`llama.h`, `ggml-backend.h`, `chat.h`,
+`json-schema-to-grammar.h`, `speculative.h`) and none to
+`llama_model_default_params` / `llama_context_default_params`, so the NIF
+source is untouched.
+
+### Changed
+
+- **RPC protocol is now major version 7** (`ggml-rpc.h`, upstream #28789). The
+  `SET_TENSOR` wire format gained a cache-flag byte, so a worker at this build
+  will refuse a client from v0.8.50 or earlier and vice versa. Rebuild both
+  ends of a two-node split together. With it comes the fix that motivated the
+  bump: `:cache_dir` on `LlamaCppEx.RPC.Server` used to hash and persist every
+  transfer over 10 MiB, including the activations `ggml_backend_sched` copies
+  between backends, which filled the worker's disk during prefill on a split
+  model. The cache now covers weight tensors only, which is what its docs
+  always described.
+- Upstream fixes worth knowing about in this range: a heap overflow in the ggml
+  CPU backend's rope work buffer caused by a `CACHE_LINE_SIZE` mismatch between
+  the C and C++ translation units (#28882 — the precompiled header is now off);
+  `common_speculative` skips the target decode when `llama_n_rs_seq` says it
+  is not needed (#28749); Maple 20B-A1B ternary MoE architecture (CPU only,
+  #27000); Metal flash-attention kernels for MiniCPM3's head sizes (#28599).
+
+Verified on macOS (Metal), M4 Max: default build **430 passed, 157 excluded**
+with no model; **573 passed, 14 excluded** for `--include smoke --include
+embeddings --include slow --include mtp` (Qwen3.5-0.8B-UD-Q8_K_XL,
+Qwen3-Embedding-0.6B-f16, Qwen3.5-0.8B-MTP-Q8_0); **6 passed** for
+`MTPE4BSidecarTest` under `--include mtp_sidecar` (gemma-4-E4B-it-Q4_K_M plus
+mtp-gemma-4-E4B-it-Q8_0). The Qwen 3.8 sidecar pair and `rpc_live` were not
+re-run at this build.
+
 ## v0.8.50
 
 ### Fixed
diff --git a/Makefile b/Makefile
index ddad3c3..aa376f1 100644
--- a/Makefile
+++ b/Makefile
@@ -36,7 +36,7 @@ endif
 # Pinned llama.cpp commit, used when vendor/llama.cpp has to be cloned. MUST
 # match the vendor/llama.cpp submodule; bump both together, see
 # docs/release-guide.md. Override to build the NIF against another revision.
-LLAMA_COMMIT ?= b6b003d2cb29647d968302eb2db8da6f66303b3e
+LLAMA_COMMIT ?= 72b590d65f04adabbb6403d75188edc77bc5a867
 
 # The commit actually on disk. A submodule can be bumped without LLAMA_COMMIT
 # following it, and the build has to key off what is really there.
diff --git a/lib/llama_cpp_ex/rpc/server.ex b/lib/llama_cpp_ex/rpc/server.ex
index 4bcfe68..bce51f6 100644
--- a/lib/llama_cpp_ex/rpc/server.ex
+++ b/lib/llama_cpp_ex/rpc/server.ex
@@ -23,7 +23,7 @@ defmodule LlamaCppEx.RPC.Server do
 
   ## The tensor cache is worth having
 
-  `:cache_dir` enables upstream's content-addressed cache for tensors over
+  `:cache_dir` enables upstream's content-addressed cache for weight tensors over
   10 MiB. Without it every model load re-pushes the whole remote share across the
   network; with it a warm load is close to free. There is no default — pass a
   path or accept the cost knowingly.
diff --git a/mix.exs b/mix.exs
index e1dea16..79a2d9d 100644
--- a/mix.exs
+++ b/mix.exs
@@ -133,7 +133,7 @@ end
 defmodule LlamaCppEx.MixProject do
   use Mix.Project
 
-  @version "0.8.50"
+  @version "0.8.51"
   @source_url "https://github.com/nyo16/llama_cpp_ex"
 
   def project do
diff --git a/vendor/llama.cpp b/vendor/llama.cpp
index b6b003d..72b590d 160000
--- a/vendor/llama.cpp
+++ b/vendor/llama.cpp
@@ -1 +1 @@
-Subproject commit b6b003d2cb29647d968302eb2db8da6f66303b3e
+Subproject commit 72b590d65f04adabbb6403d75188edc77bc5a867