feat: add browser-native text-to-speech for assistant replies - #113
Merged
Merged
Conversation
Allows hearing assistant messages aloud using browser-native SpeechSynthesis: - Speaker action button on completed assistant replies matching sibling action buttons - Client-side SpeechSynthesis hook with garbage-collection retention and cross-component state sync - Markdown/code-fence sanitization so code blocks and syntax are not read aloud - Auto-read toggle and speech voice selector in Settings -> General - Full i18n localization for English, Japanese, and Simplified Chinese
- Use addEventListener for voiceschanged so per-hook mounts stop clobbering the shared handler - Mount the speech hook once via SpeechSynthesisProvider; transcript rows consume useSpeechContext - Autoplay reads streamState.streamingMessage with assistant-role narrowing instead of stale messagesRef - TTS settings render disabled with an explanation on unsupported browsers instead of hiding - Add settingsConfig.ttsNotSupported localization (en/ja/zh-CN)
- speech sanitizer strips tag-shaped HTML only, so "x < y and z > 0" survives - utterance lifecycle callbacks identity-check the active utterance before mutating shared speech state - autoplay speaks from the render that commits the finished reply instead of refs that lag onAgentEnd - the provider owns the single speech controller; useSpeechContext no longer mounts one fallback controller per assistant row
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds browser-native text-to-speech (TTS) playback for assistant replies in ompweb using the standard Web Speech API (
window.speechSynthesis).Features
MessageView, matching sibling Copy and Fork button styling (.message-copy-action,var(--text-muted)).lib/speech-sanitizer.ts): Strips fenced code blocks, inline code syntax, raw URLs, and markdown formatting before speech synthesis so spoken output sounds natural.hooks/useSpeechSynthesis.ts): Handles client lifecycle, ChromiumSpeechSynthesisUtterancegarbage-collection retention, cross-component state synchronization via custom events, and immediate cancellation onEscapeor clicking stop.components/SettingsConfig.tsx):components/ChatWindow.tsx): Automatically triggers playback for completed turns when auto-read is enabled.en.json), Japanese (ja.json), and Simplified Chinese (zh-CN.json).Verification
lib/speech-sanitizer.test.mjs(6/6 passing).tsc --noEmitandnpm run lintclean (0 errors).