Skip to content

llama: name fused GDN outputs before callbacks - #24298

Closed
bogdanr wants to merge 1 commit into
ggml-org:masterfrom
bogdanr:fix-fgdn-result-names
Closed

llama: name fused GDN outputs before callbacks#24298
bogdanr wants to merge 1 commit into
ggml-org:masterfrom
bogdanr:fix-fgdn-result-names

Conversation

@bogdanr

@bogdanr bogdanr commented Jun 8, 2026

Copy link
Copy Markdown

Summary

Name fused GDN result tensors before running the graph callback.

Context

The FGDN support check looks for graph nodes named with the __fgdn_ar__-<layer> / __fgdn_ch__-<layer> prefix. With Gemma E2B I saw that check hit a generic name (node_47) on the CPU path, which made the support check fail before the fused path could be used.

The callback already gets the same name, but this makes the name explicit on the ggml_gated_delta_net result before the callback runs.

Notes

This did not show a measurable CPU speedup in my local Gemma E2B benchmark, and Vulkan was already enabling the fused path in my setup. The reason for the change is to keep the tensor naming consistent for the support check, not to claim a performance win.

Testing

cmake -S . -B /tmp/llama-pr-build \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_EXAMPLES=OFF \
  -DLLAMA_BUILD_SERVER=OFF \
  -DGGML_VULKAN=OFF \
  -DGGML_CUDA=OFF \
  -DGGML_METAL=OFF
cmake --build /tmp/llama-pr-build --target llama -- -j2

I also checked the patch in a local llama-cpp-rs build with Gemma E2B. Fused GDN enabled cleanly there, but I did not observe a meaningful performance change.

@bogdanr
bogdanr requested a review from CISC as a code owner June 8, 2026 10:51
@github-actions github-actions Bot added the model Model specific label Jun 8, 2026
@bogdanr
bogdanr force-pushed the fix-fgdn-result-names branch from d56d8f9 to 960276d Compare July 5, 2026 23:36
@bogdanr

bogdanr commented Jul 5, 2026

Copy link
Copy Markdown
Author

Hi @CISC, small nudge on this one. I've just rebased it onto latest master so it's clean and ready to go. It fixes a crash when loading hybrid Gated Delta Net models (Gemma E2B) on the CPU path, where the fused GDN check hits an unnamed node and aborts. Would really appreciate a review whenever you get a minute. Thanks!

@CISC

CISC commented Aug 2, 2026

Copy link
Copy Markdown
Member

#24646

@CISC CISC closed this Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model Model specific

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants