diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f4d9c7..0d8cf13 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,15 +1,20 @@ # Changelog -## Unreleased +## v0.8.49 + +This section also covers v0.8.44 (b10435, PR #86), v0.8.45 (b10582, #87), +v0.8.46 (b10665, #88), v0.8.47 (b10830, #89) and v0.8.48 (`4b98ab805`, #90), +which were tagged and published without a heading of their own; everything +below accumulated since v0.8.43. DGX Spark (GB10) support: a silent ARM code-generation bug fixed, the ggml RPC backend wired up so a model can span two machines, and a measured runbook for both configurations in [docs/dgx-spark.md](docs/dgx-spark.md). -llama.cpp bumped to [`b10830`](https://github.com/ggml-org/llama.cpp/releases/tag/b10830) -(`465e49b9c`, upstream v0.4.0), by way of b10435, b10582 and b10665, which -brought Qwen 3.8 in under the existing `qwen35` architecture, and MTP support -for its target/sidecar split (see Added). +llama.cpp bumped to [`b10944`](https://github.com/ggml-org/llama.cpp/releases/tag/b10944) +(`b6b003d2c`), by way of b10435, b10582, b10665, b10830 (upstream v0.4.0) and +`4b98ab805`, which brought Qwen 3.8 in under the existing `qwen35` +architecture, and MTP support for its target/sidecar split (see Added). Verified on macOS (Metal) at `e85caa81e`, running **every** tag the suite excludes by default. Default build: **428 passed, 149 excluded** with no model; @@ -33,6 +38,12 @@ Re-verified at `465e49b9c` (b10830) on macOS (Metal), M1 Max: default build Qwen3.6-35B-A3B-MTP-UD-Q4_K_XL); **434 passed, 143 excluded** for `--include mtp_sidecar` (Qwen3.8-27B-Q4_K_M plus its `mtp-*-Q4_0` head). +Re-verified at `b6b003d2c` (b10944) on macOS (Metal), M4 Max: default build +**428 passed, 149 excluded** with no model; **558 passed, 19 excluded** for +`--include smoke --include embeddings --include slow` +(Qwen3.5-0.8B-UD-Q8_K_XL, Qwen3-Embedding-0.6B-f16). The MTP and `rpc_live` +tags were not re-run at this build. + The one tag that is not green is `:mtp_cancel`, and it moved: see Changed. ### Fixed @@ -226,6 +237,29 @@ The one tag that is not green is `:mtp_cancel`, and it moved: see Changed. and the `ggml-cpu` CMake diff adds `iqp.cpp` and gates the SpacemiT IME kernels — nothing near the `-mcpu=native` probe. `:row` split mode still throws on CUDA (`ggml-cuda` exports no `ggml_backend_split_buffer_type`). +- **llama.cpp bumped to `b6b003d2c`** (b10944), 114 commits past b10830 by way + of `4b98ab805` (b10878+2, PR #90, which changed nothing in the binding), and + `LLAMA_COMMIT` moved with the submodule. One binding edit: #28715 renamed + `common_speculative_draft_params.n_past` to `pos0`, a pure rename (same + `llama_pos`, same meaning — the position `id_last` is decoded at), so the MTP + draft loop sets `dp.pos0`. `include/llama.h`, `ggml-backend.h` and + `ggml-rpc.h` did not change, and `llama_model_default_params()` / + `llama_context_default_params()` are value-identical. #28736 rewrote + `common/json-schema-to-grammar.h` around a new `common_chat_schema` + representation: `json_schema_to_grammar(const common_json &, bool)` — the + overload the NIF calls — is intact, `common_schema_info` (never used here) + is gone, and the error text for a rejected schema changed from + `Unrecognized schema: {…}` to `JSON schema error at #: unrecognized type + ` (likewise `Error resolving ref` → `cannot resolve $ref `). + Two grammar tests pinned that upstream wording; they now assert the error + names the offending type / `$ref`, which is the contract. `common/chat.h` + only gained `common_chat_tool_parameters`. All three defects in + [docs/release-guide.md](docs/release-guide.md) still stand, re-checked as a + source diff. Verified on macOS (Metal), M4 Max: default build **428 passed, + 149 excluded** with no model; **558 passed, 19 excluded** for + `--include smoke --include embeddings --include slow` + (Qwen3.5-0.8B-UD-Q8_K_XL, Qwen3-Embedding-0.6B-f16). The `:mtp`, + `:mtp_sidecar` and `:rpc_live` tags were not run — no MTP model was on hand. - **The `:mtp_cancel` bug no longer aborts the VM — it returns an error.** The race is unchanged and unfixed: cancellation is fire-and-forget, so reusing an `%MTP{}` session immediately after halting a stream can start decoding on diff --git a/Makefile b/Makefile index 5967683..ddad3c3 100644 --- a/Makefile +++ b/Makefile @@ -36,7 +36,7 @@ endif # Pinned llama.cpp commit, used when vendor/llama.cpp has to be cloned. MUST # match the vendor/llama.cpp submodule; bump both together, see # docs/release-guide.md. Override to build the NIF against another revision. -LLAMA_COMMIT ?= 4b98ab805a2638121f1671bf572832e07ef13e7d +LLAMA_COMMIT ?= b6b003d2cb29647d968302eb2db8da6f66303b3e # The commit actually on disk. A submodule can be bumped without LLAMA_COMMIT # following it, and the build has to key off what is really there. diff --git a/c_src/llama_cpp_ex/llama_nif.cpp b/c_src/llama_cpp_ex/llama_nif.cpp index 46343d2..e629541 100644 --- a/c_src/llama_cpp_ex/llama_nif.cpp +++ b/c_src/llama_cpp_ex/llama_nif.cpp @@ -2368,7 +2368,7 @@ fine::Ok<> generate_mtp_tokens( auto& dp = common_speculative_get_draft_params(sp.spec, seq_id); dp.drafting = true; dp.n_max = n_draft; - dp.n_past = n_past; + dp.pos0 = n_past; dp.id_last = sampled; dp.prompt = &prompt; dp.result = &drafts; diff --git a/docs/release-guide.md b/docs/release-guide.md index 1845fe0..73557d2 100644 --- a/docs/release-guide.md +++ b/docs/release-guide.md @@ -115,6 +115,15 @@ buffer's `set_tensor_2d`/`get_tensor_2d` hooks are still `NULL`. The `ggml-cpu/CMakeLists.txt` diff adds `iqp.cpp` and gates the SpacemiT IME kernel sources; the `-mcpu=native` probe is untouched. +Re-checked at `b6b003d2c` (b10944), covering the gap from `465e49b9c` in one +source diff: still all three. `ggml-rpc.cpp` was not touched at all — the RPC +buffer's `set_tensor_2d`/`get_tensor_2d` hooks are still `NULL` — and the +`ggml-cuda.cu` diff is #28079 (`GGML_FA_QUANTS`), #28604 (HIP `prop.integrated` +revert) and #26454 (gfx90c), none of them near `ggml_backend_cuda_comm_init`. +The `ggml-cpu/CMakeLists.txt` diff is #28091 (PCH and unity build, with GCC PCH +gated to x86) and #28667 (s390x `repack.cpp`); the `-mcpu=native` probe is +untouched. + | # | Upstream defect | Our workaround | Still needed? | |---|---|---|---| | 1 | `GGML_NATIVE=ON` makes ggml's `-mcpu=native` probe resolve to **base ARMv8-A** on Cortex-X925/A725 with GCC 13.3 — silently, with a soft CMake warning and exit 0. Costs every `sdot`/`smmla`/SVE kernel. | `LLAMA_CPU_ARM_ARCH` + `LLAMA_CUDA_ARCH` in the `Makefile`, which must be set together. See [DGX Spark](dgx-spark.md) and [Cross-Platform Builds](cross-platform-builds.md). | `scripts/spark/verify-build-flags.sh` on an aarch64 host. If a default build (no `LLAMA_CPU_ARM_ARCH`) now reports non-zero `sdot`/`smmla`, upstream fixed the probe. | diff --git a/lib/llama_cpp_ex/model.ex b/lib/llama_cpp_ex/model.ex index 3325cdf..886d974 100644 --- a/lib/llama_cpp_ex/model.ex +++ b/lib/llama_cpp_ex/model.ex @@ -80,7 +80,7 @@ defmodule LlamaCppEx.Model do `:tensor_split` and `:main_gpu` would index a list you never saw. With `:devices` set, they index this one. - > #### Split modes at llama.cpp b10830 (`465e49b9`) {: .warning} + > #### Split modes at llama.cpp b10944 (`b6b003d2`) {: .warning} > > `:layer` splits contiguous layer ranges across devices, one KV cache per > device, and is the only mode that works across hosts. diff --git a/mix.exs b/mix.exs index b548849..eb67dbb 100644 --- a/mix.exs +++ b/mix.exs @@ -133,7 +133,7 @@ end defmodule LlamaCppEx.MixProject do use Mix.Project - @version "0.8.43" + @version "0.8.49" @source_url "https://github.com/nyo16/llama_cpp_ex" def project do diff --git a/test/llama_cpp_ex_test.exs b/test/llama_cpp_ex_test.exs index 7a39b9d..1da2d62 100644 --- a/test/llama_cpp_ex_test.exs +++ b/test/llama_cpp_ex_test.exs @@ -35,8 +35,11 @@ defmodule LlamaCppExTest do # There used to be a bare `rescue _ -> :ok` after this assert_raise, which # swallowed the ExUnit.AssertionError it raises on failure — the test could # not fail, in either direction. + # Assert on the offending type name, not llama.cpp's phrasing: upstream + # reworded this from "Unrecognized schema" to "unrecognized type" in + # b10944 and the wording is not part of our contract — naming the culprit is. assert_raise ArgumentError, - ~r/failed to convert JSON schema to grammar.*Unrecognized schema/s, + ~r/failed to convert JSON schema to grammar.*invalid_type_that_does_not_exist/s, fn -> LlamaCppEx.Grammar.from_json_schema!(%{ "type" => "invalid_type_that_does_not_exist" @@ -62,12 +65,12 @@ defmodule LlamaCppExTest do LlamaCppEx.Grammar.from_json_schema(%{"type" => "not_a_json_schema_type"}) assert is_binary(reason) - assert reason =~ "Unrecognized schema" + assert reason =~ "not_a_json_schema_type" assert {:error, ref_error} = LlamaCppEx.Grammar.from_json_schema(%{"$ref" => "#/definitions/missing"}) - assert ref_error =~ "Error resolving ref" + assert ref_error =~ "#/definitions/missing" end test "from_json_schema refuses schemas that would blow the C++ stack" do diff --git a/vendor/llama.cpp b/vendor/llama.cpp index 4b98ab8..b6b003d 160000 --- a/vendor/llama.cpp +++ b/vendor/llama.cpp @@ -1 +1 @@ -Subproject commit 4b98ab805a2638121f1671bf572832e07ef13e7d +Subproject commit b6b003d2cb29647d968302eb2db8da6f66303b3e