Skip to content

feat(voices): voice picker with audio preview in the generate box - #1200

Open
joshfrenette22-stack wants to merge 2 commits into
jamiepine:mainfrom
joshfrenette22-stack:feature/voice-picker-preview
Open

joshfrenette22-stack wants to merge 2 commits into
jamiepine:mainfrom
joshfrenette22-stack:feature/voice-picker-preview

Conversation

@joshfrenette22-stack

Copy link
Copy Markdown

What

A searchable voice picker in the generate box, with a ▶ button on every voice that plays a short sample phrase before you generate. Built-in Kokoro and Qwen CustomVoice voices are listed alongside your own voices, so you can audition them without leaving the generate box.

Also fixes a Kokoro bug that made every voice you hadn't used before fail on generation (second commit below).

Why

Right now the generate box has no way to choose a voice (only the Stories tab had a plain dropdown), and the Create Voice dialog's built-in voice grid can't be auditioned. You pick a name from a list of 54 Kokoro voices blind, create a profile, and only then hear it.

How it works

Picker (VoicePicker.tsx)

  • A pill in the generate box's bottom row opens a popover with a search field, a "My voices" section, and a section per built-in engine (Kokoro, Qwen CustomVoice).
  • Choosing one of your voices selects it. The existing profile-sync effect then switches engine and language.
  • Choosing a built-in voice reuses the matching preset profile if there is one, otherwise creates it (name de-duplicated, since profile names are unique) and selects it. Generation is always tied to a profile, so this avoids a separate "add voice" step.
  • Replaces the plain voice Select that only the Stories tab had, so the showVoiceSelector prop is gone.

Preview playback (useVoicePreview, VoicePreviewButton)

  • One clip plays at a time. Clips are cached as object URLs for the life of the hook, so replays are instant, and a slow response for an earlier click won't talk over a newer one.
  • The Create Voice dialog's built-in grid gets the same buttons.

Backend

  • GET /profiles/presets/{engine}/{voice_id}/preview: renders a short phrase in the voice's own language (11 phrases; unknown languages fall back to English) with a fixed seed and caches the WAV under the cache dir. Cache file names are digests, so a voice id never shapes a path.
  • GET /profiles/{profile_id}/preview: same for any profile. Built-in profiles share the preset cache; cloned and designed voices render on demand (not disk-cached, since editing the profile changes the sound).
  • Previews run one at a time behind a lock, since they share the model with real generations.
  • A model that isn't downloaded returns a 409 pointing at the Models tab, instead of the stock message that refers to /generate.

Kokoro fix (separate commit)

fix(kokoro): the model download only fetches the weights; each voice is a separate file that KPipeline pulls from the Hub on first use. huggingface_hub keeps one HTTP session per thread, and a session built during an offline window (opened while loading a cached model) keeps its offline adapter. A thread reused for generation then fails with OfflineModeIsEnabled for every voice that hasn't been used before. This reproduced on a plain generation, not just previews. The fix checks the voice file is cached and, if not, fetches it inside a network-allowed window with fresh sessions.

Testing

  • New backend tests (13): phrase coverage for every built-in voice language, cache path stability, render-once-then-cache, 404s, the missing-model message, and the Kokoro voice fetch. The renderer and Hub are stubbed, so no model is loaded.
  • tsc passes for app and web. Biome reports nothing in the new files (remaining findings are in lines I didn't touch).
  • Run against a real backend:
    • Kokoro previews for voices that weren't cached (af_bella, bm_george) rendered and downloaded the voice on demand. Transcribing the clips with the app's Whisper returned the exact phrase.
    • A cloned profile (Qwen base) previewed end to end and transcribed back to the phrase; the throwaway profile was deleted afterwards.
    • Opened the picker in the app and checked preview, selecting a built-in voice (profile created, engine and language switched), and search.
    • In the Create Voice dialog's built-in grid, ▶ plays a voice (including one whose file wasn't downloaded yet) without selecting its card.

Not covered / notes

  • UI checks were done in a browser against the Vite dev server, not the packaged Tauri build, and there are no screenshots attached.
  • Qwen CustomVoice previews only had the "model not downloaded" (409) path exercised, and designed-voice previews weren't run (no VoiceDesign model on hand).
  • Only English strings were added; other locales fall back to English.
  • backend/tests/test_offline_guard.py::test_nesting_opposite_mode_on_same_thread_raises_instead_of_deadlocking fails on current main too, independent of this change.

🤖 Generated with Claude Code

joshfrenette22-stack and others added 2 commits October 13, 2026 09:58
The model download only fetches the Kokoro weights; each voice is a
separate file that KPipeline pulls from the Hub the first time the voice is
used. huggingface_hub keeps one HTTP session per thread, and a session built
inside an offline window (opened while loading a cached model) keeps its
offline adapter. A thread reused for generation then failed with
OfflineModeIsEnabled for every voice that had not been used before.

Check the voice file is cached before generating, and if not, fetch it
inside a network-allowed window with fresh sessions.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add a searchable voice picker to the generate box that lists your voices
and every built-in Kokoro and Qwen CustomVoice voice, each with a play
button that speaks a short sample phrase before you generate. Choosing a
built-in voice adds it as a preset profile (generation is always tied to a
profile) and selects it. The Create Voice dialog's built-in voice grid gets
the same preview buttons.

Backend:
- GET /profiles/presets/{engine}/{voice_id}/preview renders the sample
  phrase in the voice's own language and caches it on disk.
- GET /profiles/{profile_id}/preview does the same for any profile, so
  cloned and designed voices can be heard too.
- A missing model returns a 409 that points at the Models tab.

The picker replaces the plain voice dropdown that only the Stories tab had.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant