Skip to content

DSpark draft model fails to load on Windows Vulkan build: "invalid vector subscript" #101

Description

@Zolikabacsi

Summary

Loading the DSpark speculative-decoding drafter (Ternary-Bonsai-27B-dspark-Q4_1.gguf) via --model-draft fails on the Windows Vulkan prebuilt binary with invalid vector subscript. The same command works on the Windows CPU prebuilt binary from the same release.

Environment

  • Release: prism-b9596-9fcaed7
  • Binary: llama-bin-win-vulkan-x64.zip (reports build: b1-9fcaed7, MSVC 19.51)
  • OS: Windows x64
  • GPU: AMD Radeon RX 6800S (8 GB), Adrenalin driver, Vulkan backend
  • Main model: Ternary-Bonsai-27B-Q2_0.gguf from prism-ml/Ternary-Bonsai-27B-gguf (loads and runs fine on the same Vulkan build without the drafter, ~17 tok/s)
  • Draft model: Ternary-Bonsai-27B-dspark-Q4_1.gguf from the same repo (sha256 c4810091d244eddc61a0cc4966e584b0959f141e3c66c0d371a6652d9f647da9)

Reproduction

llama-cli.exe -m Ternary-Bonsai-27B-Q2_0.gguf ^
  --model-draft Ternary-Bonsai-27B-dspark-Q4_1.gguf ^
  --spec-type draft-dspark --spec-draft-n-max 4 ^
  -p "Hi" -n 8 --temp 0.7 --top-p 0.95 --top-k 20 -c 4096 -ngl 99

Observed (Vulkan build)

0.08.716.940 E llama_model_load: error loading model: invalid vector subscript
0.08.716.949 E llama_model_load_from_file_impl: failed to load model
0.08.717.101 E srv    load_model: failed to load draft model, 'Ternary-Bonsai-27B-dspark-Q4_1.gguf'

The failure happens while loading the draft model; the main model loads successfully on this build.

Expected / Control

The identical command using llama-bin-win-cpu-x64.zip from the same release (build b9596-9fcaed763, Clang 20.1.8) loads both models and generates correctly with speculative decoding enabled.

Notes

  • The drafter's GGUF metadata reports general.architecture = dspark with tokenizer.ggml.model = none (tokenizer shared with the target), in case the draft-model load path on the Vulkan build handles this differently.
  • Happy to provide full --verbose logs or test a patched build.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions