Skip to content

ggml : report allocation failure from the meta buffer type - #64

Open
Piggidragon wants to merge 3 commits into
GenerelSchwerz:llama/devfrom
Piggidragon:ggml/meta-alloc-failure
Open

ggml : report allocation failure from the meta buffer type#64
Piggidragon wants to merge 3 commits into
GenerelSchwerz:llama/devfrom
Piggidragon:ggml/meta-alloc-failure

Conversation

@Piggidragon

Copy link
Copy Markdown

Overview

ggml_backend_meta_buffer_type_alloc_buffer and ggml_backend_meta_alloc_ctx_tensors_from_buft
asserted when one of the simple buffers behind the meta buffer could not be allocated. Every other
buffer type returns NULL there, and callers such as llama_params_fit rely on that to probe how
much fits. Under split mode tensor an out-of-memory condition aborted the process instead.

Free the buffers already allocated and return NULL. The second function also has to clear the
tensors it already pointed at the buffer it frees.

Split out of #57, which needed the meta backend to survive a failed probe. No behaviour changes
while allocation succeeds.

Testing

test-llama-archs -s 1 with 1, 2, 3 and 4 virtual CUDA devices: passes.
Built with -DLLAMA_FATAL_WARNINGS=ON.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES - implemented by an agent on my instruction, see the commit trailer.

@GenerelSchwerz

Copy link
Copy Markdown
Owner

Automated preliminary review by Codex; the repository owner plans a separate manual review.

Verdict: CONDITIONAL PASS.

Blocking:

  • None. I found no blocking correctness, security, performance, complexity, or pristine-upstream compatibility issue.

Will slow review:

  • ggml/src/ggml-backend-meta.cpp:1703-1708 and ggml/src/ggml-backend-meta.cpp:1766-1773 add allocation-failure cleanup without focused failure-injection coverage. Reuse the allocation-failure hook pattern at tests/test-llama-archs.cpp:414-430 and its retry checks at tests/test-llama-archs.cpp:675-692 to prove partial allocation cleanup, a NULL return, cleared buffer and data on every original tensor, and a successful retry.

Nits:

  • None.

Developmental progress: The broad precursor scope has been reduced to this focused single-file change, and the original dangling-pointer problem is fixed. Unrelated predecessor concerns were split into other PRs.

Reviewed head: a18bcdf

The meta buffer type asserted when a simple buffer could not be allocated, so an
out-of-memory condition aborted instead of returning NULL like every other buffer
type. Free what was allocated and return NULL.

Assisted-by: Claude Opus 5
@Piggidragon
Piggidragon force-pushed the ggml/meta-alloc-failure branch from 1ccc9cc to e5bbd44 Compare September 6, 2026 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants