Doesn't work adreno gpus (Qualcomm Adreno Vulkan)

[guranu]

Hello, i compiled audio.cpp with vulkan support. It compiled fine but when trying VoxCPM 2 (or any other model) it fails with this error:
ggml_vulkan: Compute pipeline creation failed for mul_mat_vec_q8_0_q8_1_f32 ggml_vulkan: vk::Device::createComputePipeline: ErrorUnknown Segmentation fault

From my research the error says that mul_mat_vec_q8_0_q8_1_f32 isn't supported on my device, but other implementations such as llama.cpp-omni (the official implementation of voxcpm 2 with gguf support) and VoxCPM.cpp: https://github.com/bluryar/VoxCPM.cpp can run the model without any problem.

Suggestion on how to "fix" this:
Provide Q4_0/Q5_0 variants for each model.

Logs:

~/.../build/bin $ sudo ./audiocpp_cli --task tts --family voxcpm2 --model /sdcard/Download/vox_test/voxcpm2-q8_0.gguf --backend vulkan --threads 8 --text "ここは……どこ？" --voice-ref /sdcard/Download/vox_test/rimi16khz.wav --out rimi.wav
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Adreno (TM) 830 (Qualcomm Technologies Inc. Adreno Vulkan Driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: none
ggml_vulkan: Compute pipeline creation failed for mul_mat_vec_q8_0_q8_1_f32
ggml_vulkan: vk::Device::createComputePipeline: ErrorUnknown
Segmentation fault

[0xShug0]

@guranu This is a backend compatibility issue. Could you try GGML_VK_DISABLE_INTEGER_DOT_PRODUCT=1 ./audiocpp_cli

[guranu]

@guranu This is a backend compatibility issue. Could you try GGML_VK_DISABLE_INTEGER_DOT_PRODUCT=1 ./audiocpp_cli

Like rebuild it and run it again?

[0xShug0]

@guranu No need rebuild. Just set this env variable during run.

Update: similar to this issue #434 (comment)

[guranu]

Now i get this error:
audiocpp_cli failed: vk::Queue::submit: ErrorDeviceLost

[0xShug0]

@guranu The workaround was not actually applied in that screenshot ---- int dot: 1
If GGML_VK_DISABLE_INTEGER_DOT_PRODUCT=1 were active, it should report int dot: 0.. I’m not sure how to set the environment variable in your setup, so you may need to look it up for your specific environment.

[guranu]

Fixed the problem by reinstalling termux, but the output quality is worse compared to VoxCPM.cpp

audio.cpp:

rimi-hd.wav

VoxCPM.cpp:

rimi-vulkan.wav

This the command the used in audio.cpp:
./audiocpp_cli --task tts --family voxcpm2 --model /sdcard/Download/vox_test/voxcpm2-q8_0.gguf --backend vulkan --threads 8 --text "ここは……どこ？" --voice-ref /sdcard/Download/vox_test/rimi16khz.wav --out rimi.wav

[0xShug0]

There have been issues with output failures on other backends (e.g., Irodori + Intel Iris Xe + Vulkan). Qualcomm Vulkan validation is not currently in our test plan, so it’s hard to say what caused this from one subjective comparison. But please do try different seeds. VoxCPM2 is seed-sensitive.

In our development, the reference is the official Python implementation. I’m happy to reopen the issue and investigate more if you can provide logs or evidence showing that audio.cpp VoxCPM2 output quality is worse than Python consistently under controlled settings.