llama-switch

Switch between local llama.cpp models from a CLI or a modern web GUI — per-model configs, one-click switching, and live GPU status.

What's here

llama-switch	CLI. Interactive arrow-key picker, or llama-switch <model>. Writes the model's config and restarts the server.
gui/	Web control panel (Flask). Model grid, one-click switch, per-model config editor, live VRAM + decode-speed status.

How it works

llama-switch reads a models.yaml registry, writes the chosen model's settings to ~/.config/llama-server/active.env, and restarts a systemd user service (llama-server) whose start.sh sources that env file and launches llama-server. The GUI does the same over HTTP and can edit models.yaml in place.

models.yaml ──> llama-switch ──> active.env ──> start.sh ──> llama-server

Setup

1. Registry — put your .gguf files in one folder and create ~/.config/llama-server/models.yaml (see examples/models.yaml).
2. Server — install examples/start.sh to ~/.config/llama-server/start.sh and examples/llama-server.service to ~/.config/systemd/user/, then:

   systemctl --user enable --now llama-server

3. CLI — install -m755 llama-switch ~/.local/bin/, then llama-switch (picker) · llama-switch <model> · llama-switch --list.
4. GUI — cd gui && pip install -r requirements.txt && ./start-gui.sh, then open http://127.0.0.1:7860.

Model config (models.yaml)

models_dir: /path/to/your/models
models:
  my-model:
    file: My-Model-Q4_K_M.gguf   # required
    ngl: 99                      # GPU layers (99 = all)
    ctx_size: 65536              # context window
    parallel: 1                  # concurrent slots (optional)
    mmproj: mmproj-F16.gguf      # vision projector (optional)
    mtp: MTP/draft.gguf          # speculative-decoding draft (optional)
    chat_template: custom.jinja  # override template (optional)
    extra:                       # any extra llama-server flags (optional)
      - --reasoning on
      - --temp 0.7
    description: "notes shown in the CLI/GUI"

Env overrides: LLAMA_MODELS_DIR, LLAMA_GPU_CARD (sysfs path for VRAM stats), LLAMA_GUI_PORT.

Requirements

Linux · llama.cpp (llama-server on PATH) · systemd user services · Python 3.9+. GUI: Flask + PyYAML.

License

MIT