Feature Request: Support for Gemma 3 models family

[MoonRide303]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

It would be nice if we could run Gemma 3 models family in llama.cpp.

Currently conversion to GGUF fails with ERROR:hf-to-gguf:Model Gemma3ForCausalLM is not supported.

Motivation

Gemma family was supported so far.

Possible Implementation

No response

[sais-github]

There's an implementation in Ollama, which could probably(?) be ported

[MoonRide303]

There's an implementation in Ollama, which could probably(?) be ported

It's written in Go, but it might be helpful: ollama/ollama#9661. Seems to be working:

Example

> ollama run gemma3:4b
>>> Please write me 10 sentences ending with a pear.
Okay, here are 10 sentences, each ending with the word “pear.”

1.  The aroma of cinnamon and spice filled the air, perfectly complementing the dessert—a poached pear.
2.  He carefully selected a ripe, juicy fruit from the basket, a beautiful green pear.
3.  Her grandmother’s secret recipe always included a touch of honey and a slice of pear.
4.  After a long hike, a refreshing snack was a crisp, cool pear.
5.  The artist meticulously painted the still life, focusing on the delicate curve of a pear.
6.  Despite her attempts to disguise it, the sweetness of the pie was undeniably a pear.
7.  He polished the antique silverware, reflecting the light on a polished pear.
8.  The children giggled with delight as they bit into their freshly picked pear.
9.  She arranged the fruit on the table, a vibrant addition to the colorful display of a pear.
10.  Lost in thought, he reached for a comforting treat—a simple, sweet pear.

---

Would you like me to create a different set of sentences, perhaps with a specific theme or tone?

[MoonRide303]

I just tested b4875 (7841fc7 / #12343, ❤️ @ngxson) using gemma-3-4b-it-Q8_0.gguf (GGUFs @ ggml-org), and... it seems to be working, now :).

Example

> llama-cli.exe -ngl 99 -m gemma-3-4b-it-Q8_0.gguf -i
(...)
== Running in interactive mode. ==
 - Press Ctrl+C to interject at any time.
 - Press Return to return control to the AI.
 - To return control without starting a new line, end your input with '/'.
 - If you want to submit another line, end your input with '\'.
 - Not using system message. To change it, set a different value via -sys PROMPT


> Please write me 10 sentences ending with a pear.
Okay, here are 10 sentences, each ending with the word “pear”:

1.  After a long hike, all I wanted was a juicy, refreshing pear.
2.  The artist carefully arranged the still life, featuring a single, perfect pear.
3.  My grandmother always insisted on serving a sliced fruit with her afternoon tea, invariably a pear.
4.  He polished the antique bowl until it gleamed, displaying his prized possession – a pear.
5.  The scent of cinnamon and baked apples filled the kitchen, hinting at a delicious pear.
6.  She picked a ripe one from the tree, a sweet and fragrant pear.
7.  Despite the rain, the orchard was overflowing with golden, tempting pear.
8.  The children giggled as they tossed the fruit around, each clutching a small pear.
9.  Lost in thought, he stared out the window, contemplating the simple beauty of a pear.
10.  For a healthy and satisfying snack, I always reach for a crisp, juicy pear.

---

Would you like me to generate more sentences, or perhaps with a specific theme?