Skip to content

Bump llama.cpp to 72b590d65 (b10989+1, +46 from b6b003d2c), release v0.8.51 - #95

Merged
nyo16 merged 1 commit into
masterfrom
bump-llama-cpp-b10989
Sep 16, 2026
Merged

nyo16 merged 1 commit into
masterfrom
bump-llama-cpp-b10989

Conversation

@nyo16

@nyo16 nyo16 commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Bumps vendor/llama.cpp from b6b003d2c (b10944) to 72b590d65 — origin/master, one commit past b10989 (upstream v0.4.1). +46 commits. Pins
LLAMA_COMMIT in the Makefile to the same SHA and bumps @version to 0.8.51.

API compatibility

No diff in any header the NIF includes — llama.h, ggml-backend.h, chat.h, json-schema-to-grammar.h, speculative.h — and none in llama_model_default_params / llama_context_default_params.
c_src/ is untouched.

The one relevant header change is ggml-rpc.h: RPC_PROTO_MAJOR_VERSION 6 → 7 (ggml-org/llama.cpp#28789). SET_TENSOR gains a cache-flag byte, so a worker at this build refuses a v0.8.50 client and
vice versa — both ends of a two-node split must rebuild together.

The same upstream change fixes what motivated it: the RPC hash cache (:cache_dir on LlamaCppEx.RPC.Server) used to hash and persist every transfer over 10 MiB, including the activations
ggml_backend_sched copies between backends, filling the worker's disk during prefill on a split model. It now covers weight tensors only. The :cache_dir moduledoc is adjusted to say so.

Release-guide workaround #3 (ggml_backend_rpc_start_server returns void) is unchanged upstream; the preflight-bind workaround stays.

Notable upstream changes in range

  • ggml-cpu: heap overflow in the rope work buffer from a CACHE_LINE_SIZE mismatch between C and C++ TUs; precompiled header disabled (#28882)
  • common_speculative skips the target decode when llama_n_rs_seq says it is not needed (#28749)
  • Maple 20B-A1B ternary MoE architecture, CPU only (#27000)
  • Metal FA kernels for HSK=96 / HSV=64, MiniCPM3 (#28599)

Verification

macOS (Metal), M4 Max, LLAMA_BACKEND=metal source build:

Run Result
mix test (no model) 430 passed, 157 excluded
--include smoke --include embeddings --include slow --include mtp (Qwen3.5-0.8B-UD-Q8_K_XL, Qwen3-Embedding-0.6B-f16, Qwen3.5-0.8B-MTP-Q8_0) 573 passed, 14 excluded
MTPE4BSidecarTest under --include mtp_sidecar (gemma-4-E4B-it-Q4_K_M + mtp-gemma-4-E4B-it-Q8_0) 6 passed
mix format --check-formatted clean

Not re-run at this build: the Qwen 3.8 sidecar pair (MTPSidecarTest), rpc_live, dialyzer, and the Hex source-build round-trip. checksum.exs is regenerated by the precompile workflow after tagging.

Files

  • vendor/llama.cpp — submodule → 72b590d65
  • MakefileLLAMA_COMMIT
  • mix.exs@version 0.8.50 → 0.8.51
  • CHANGELOG.md — v0.8.51 section
  • lib/llama_cpp_ex/rpc/server.ex:cache_dir doc: "weight tensors"

…0.8.51

No NIF-facing header changed; RPC protocol major goes 6 -> 7 (SET_TENSOR
cache-flag byte), so both ends of a two-node split must rebuild together.
The same upstream change stops the RPC hash cache persisting activations.

Verified on Metal (M4 Max): 430 passed default, 573 passed with
smoke/embeddings/slow/mtp, 6 passed MTPE4BSidecarTest.
@nyo16
nyo16 merged commit 0d1c47b into master Sep 16, 2026
9 checks passed
@nyo16
nyo16 deleted the bump-llama-cpp-b10989 branch September 16, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant