[delhalflorian-web]

What

Adds Boson AI's Higgs Audio v3 4B (bosonai/higgs-tts-3-4b) as a new higgs engine, MLX only (Apple Silicon). It runs on mlx-audio's higgs_audio_v3 port: zero-shot voice cloning across the 23 languages Voicebox already lists, inline control tags (<|emotion:amusement|>, <|sfx:laughter|>…), 24 kHz output. On an M4 Max it renders at roughly realtime (RTF ≈ 0.9).

Followed .agents/skills/add-tts-engine / docs/.../tts-engines.mdx, Phases 0–5.

Changes

• Backend: new backend/backends/higgs_mlx_backend.py, modelled on chatterbox_mlx_backend (deferred file-path voice prompt, shared MLX worker thread, model_load_progress). It is registered in backends/__init__.py, and the ModelConfig is only emitted when get_backend_type() == "mlx". On other platforms the factory raises a clear error.
• API / services: higgs added to the engine regex in models.py and to CLONING_ENGINES in services/profiles.py. Without the latter, every generation from a cloned profile is rejected.
• Frontend: engine union types, ENGINE_LANGUAGES, EngineModelSelector (options, description, cloning set), useGenerationForm (schema + model/display name), ModelManagement, ProfileForm, format.ts. Labelled "non-commercial".
• Dependencies: mlx-audio 0.4.1 → 0.5.9 (still --no-deps, in justfile, release.yml and the requirements-mlx.txt note); mlx>=0.31.1. No new packages. Dockerfile is untouched (no MLX on Linux).
• PyInstaller: hidden imports for the backend and mlx_audio.tts.models.higgs_audio_v3, which is loaded by name. The rest is already covered by --collect-submodules mlx_audio.
• Tests: backend/tests/test_higgs_mlx_backend.py (registry wiring + generate + clone + unload; skipped off MLX or without cached weights) and a higgs row in the e2e matrix.

Things reviewers should know

1. mlx-audio 0.5.9 upstream bug: when load() gets a local snapshot path, the higgs_multimodal_qwen3 → higgs_audio_v3 remap is skipped, because the stage-2 name-part matching overrides it. The backend passes model_type="higgs_audio_v3" explicitly.
2. Whisper regression from the bump, fixed here: with 0.5.9, MLX Whisper's temperature fallback turned a short French clip into multilingual gibberish after the first sentence, while 0.4.1 transcribed it cleanly. MLXSTTBackend now passes temperature=0.0, which matches the 0.4.1 output (verified with and without language).
3. Why not the 4-bit MLX artifact (Reza2kn/Higgs-Audio-v3-TTS-4bit-MLX): its safetensors header contains only the 756 quantized body.layers.* tensors (qweight/scales/biases). There is no audio codec, embeddings or final norm, and the index points at a model.safetensors that isn't in the repo. It can't be loaded, so the engine downloads the 9.3 GB bf16 checkpoint.
4. Licence: the weights are under Boson's Higgs Audio v3 research and non-commercial licence. The engine is labelled as such and is never a default.
5. Platform UI: the frontend has no per-platform engine filtering, so the option is visible everywhere. Off Apple Silicon, there is no model entry and the backend returns a clear error.

Testing (macOS 26, M4 Max, Python 3.12, transformers 4.57.3)

• Phase 0 in Voicebox's own venv with mlx-audio 0.5.9: French generation and cloning OK. Existing MLX call sites re-run on 0.5.9: Qwen3-TTS 1.7B clone OK, Whisper OK (after the fix), Chatterbox MLX signature checked only (weights not cached).
• pytest backend/tests/test_higgs_mlx_backend.py: 2 passed.
• Frozen binary (just build-server) through test_all_models_e2e.py --only higgs,qwen: higgs passed, qwen 1.7B passed.
• tsc --noEmit clean; Biome / Ruff introduce no new findings in touched lines (pre-existing ones left alone to keep the diff small).
• Full .app bundle built and launched locally.

Not tested: download from a clean HF cache in the frozen build (the 9.3 GB model was already cached), Chatterbox MLX real generation, Windows/Linux (engine is MLX-only).

Pre-existing, unrelated: in the e2e run, qwen 0.6B (not cached) fails with OfflineModeIsEnabled right after the cached 1.7B run. The same failure reproduces on a 0.6.0 build with mlx-audio 0.4.1, so it predates this PR (offline guard in hf_offline_patch). I'm investigating it separately.

🤖 Generated with Claude Code