Skip to content

Bump llama.cpp to b10830 (465e49b9c, upstream v0.4.0) - #89

Merged
nyo16 merged 1 commit into
masterfrom
bump-llama-cpp-b10830
Sep 7, 2026
Merged

nyo16 merged 1 commit into
masterfrom
bump-llama-cpp-b10830

Conversation

@nyo16

@nyo16 nyo16 commented Sep 7, 2026

Copy link
Copy Markdown
Owner

165 commits past b10665 (ca3d5a3e1). LLAMA_COMMIT in the Makefile moves
with the submodule. No NIF source change.

API surface

  • include/llama.h: only llama_tensor_read_lazyllama_lazy_mode
    (tensor_read_lazylazy_mode in llama_model_params, #27969). The
    NIF neither sets nor reads it.
  • common/common.h: same rename plus preserve_reasoning_specified.
  • ggml-backend.h, ggml-rpc.h, chat.h, json-schema-to-grammar.h,
    speculative.h: unchanged.
  • llama_model_default_params() / llama_context_default_params():
    value-identical to b10665 apart from the rename.

Upstream defects we work around — all three still stand (source diff)

  • ggml_backend_rpc_start_server still returns void.
  • ggml_backend_cuda_comm_init untouched; RPC set/get_tensor_2d hooks
    still NULL. RPC diff is #26500 (cross-server buffer serialisation) and
    #27960 (ggml_op_alloc_size_may_expand).
  • ggml-cpu/CMakeLists.txt diff adds iqp.cpp and gates SpacemiT IME
    sources; the -mcpu=native probe is untouched.
  • :row split mode still throws on CUDA (no ggml_backend_split_buffer_type).

Behaviour change worth knowing

  • TENSOR_READ_LAZY tensors (Gemma 4 per-layer embeddings, Qwen4-exp PLE)
    now map the file regardless of load_mode (#27837). :use_mlock alone
    no longer guarantees "no mapping" for those two tensor kinds; every other
    tensor behaves as Model.load/2 documents.

Verification (macOS, Metal, M1 Max, LLAMA_BACKEND=metal)

Run Result
mix test (no model) 428 passed, 149 excluded
--include smoke embeddings slow mtp (Qwen3.5-0.8B-UD-Q4_K_XL, Qwen3-Embedding-0.6B-f16, Qwen3.6-35B-A3B-MTP-UD-Q4_K_XL) 569 passed, 8 excluded
--include mtp_sidecar (Qwen3.8-27B-Q4_K_M + mtp-Qwen3.8-27B-Q4_0) 434 passed, 143 excluded
--only mtp_cancel fails as documented: verify decode failed: code=-1, no abort

mix compile --warnings-as-errors and mix format --check-formatted clean.

Not run: rpc_live (needs an RPC build and a worker); the Hex tarball
source-build check.

Files

  • vendor/llama.cpp465e49b9cea78a68b9c244ffb48d0ee24a82873d
  • Makefile: LLAMA_COMMIT
  • CHANGELOG.md: bump bullet, re-verification counts, mtp_cancel state
  • docs/release-guide.md: defect re-check at b10830
  • lib/llama_cpp_ex/model.ex: split-mode pin reference

165 commits past b10665 (ca3d5a3e1). LLAMA_COMMIT in the Makefile moves
with the submodule. No NIF source change.

API surface
- include/llama.h: only llama_tensor_read_lazy -> llama_lazy_mode
  (tensor_read_lazy -> lazy_mode in llama_model_params, #27969). The NIF
  neither sets nor reads it.
- common/common.h: same rename plus preserve_reasoning_specified.
- ggml-backend.h, ggml-rpc.h, chat.h, json-schema-to-grammar.h,
  speculative.h: unchanged.
- llama_model_default_params() / llama_context_default_params():
  value-identical to b10665 apart from the rename.

Upstream defects we work around: all three still stand (source diff)
- ggml_backend_rpc_start_server still returns void.
- ggml_backend_cuda_comm_init untouched; RPC set/get_tensor_2d hooks still
  NULL. RPC diff is #26500 (cross-server buffer serialisation) and #27960
  (ggml_op_alloc_size_may_expand).
- ggml-cpu/CMakeLists.txt diff adds iqp.cpp and gates SpacemiT IME sources;
  the -mcpu=native probe is untouched.
- :row split mode still throws on CUDA (no ggml_backend_split_buffer_type).

Behaviour change worth knowing
- TENSOR_READ_LAZY tensors (Gemma 4 per-layer embeddings, Qwen4-exp PLE)
  now map the file regardless of load_mode (#27837).

Verification (macOS, Metal, M1 Max, LLAMA_BACKEND=metal)
- mix test (no model): 428 passed, 149 excluded
- --include smoke embeddings slow mtp (Qwen3.5-0.8B-UD-Q4_K_XL,
  Qwen3-Embedding-0.6B-f16, Qwen3.6-35B-A3B-MTP-UD-Q4_K_XL):
  569 passed, 8 excluded
- --include mtp_sidecar (Qwen3.8-27B-Q4_K_M + mtp-Qwen3.8-27B-Q4_0):
  434 passed, 143 excluded
- --only mtp_cancel: fails as documented, "verify decode failed: code=-1",
  no abort
- mix compile --warnings-as-errors and mix format --check-formatted clean

Not run: rpc_live (needs an RPC build and a worker); the Hex tarball
source-build check.
@nyo16
nyo16 merged commit 26b9ff0 into master Sep 7, 2026
9 checks passed
@nyo16
nyo16 deleted the bump-llama-cpp-b10830 branch September 7, 2026 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant