This is going to be a project to overhaul how TTS works in RealtimeChat. Today, it works by attempting to keep the main activity alive while the active channel is streaming. This comes with some complications, namely:
It's hard to keep that activity alive
It's battery intensive since the activity runs other things
It's hard to ensure message parity since the message history fetches things that should not be read
TTS logic is intermingled with other logic
It's not so easy to use native APIs to customize TTS voices as we are reliant on third-party packages, so we run into issues like this: [tts] tts does not work when app is in the background on ios #388
This project will have four components:
The iOS TTS engine
The Android TTS engine
The TTS circular buffer
The TTS isolate
The Android and iOS TTS engines will have the same Dart APIs following:
class Engine:
getLanguages(): string[]
getVoices(languageId: string): string[]
speak({ text: string, languageId: string, voiceId: string }): Promise<void>
stop() // immediately stop speaking
### Milestone 1: Native APIs
- [ ] https://github.com/muxable/rtchat/issues/1080
- [ ] https://github.com/muxable/rtchat/issues/1081
- [ ] https://github.com/muxable/rtchat/issues/1083
- [ ] https://github.com/muxable/rtchat/issues/1086
- [ ] https://github.com/muxable/rtchat/issues/1087
- [ ] https://github.com/muxable/rtchat/issues/1088
- [ ] https://github.com/muxable/rtchat/issues/1092
- [ ] https://github.com/muxable/rtchat/issues/1093
- [ ] https://github.com/muxable/rtchat/issues/1097
### Milestone 2: TTS circular buffer
- [ ] https://github.com/muxable/rtchat/issues/1098
- [ ] https://github.com/muxable/rtchat/issues/1099
- [ ] https://github.com/muxable/rtchat/issues/1100
- [ ] https://github.com/muxable/rtchat/issues/1101
- [ ] https://github.com/muxable/rtchat/issues/1102
### Milestone 4: Running in an isolate
- [ ] https://github.com/muxable/rtchat/issues/1103
- [ ] https://github.com/muxable/rtchat/issues/1104
- [ ] https://github.com/muxable/rtchat/issues/1105
- [ ] https://github.com/muxable/rtchat/issues/1106
- [ ] https://github.com/muxable/rtchat/issues/1151
- [ ] https://github.com/muxable/rtchat/issues/1152
### Milestone 5: Migrating the new TTS system
- [ ] https://github.com/muxable/rtchat/issues/1153
- [ ] https://github.com/muxable/rtchat/issues/1154
### Milestone 6: Testing and cleanup
- [ ] https://github.com/muxable/rtchat/issues/1189
- [ ] Ship a new release with the new TTS engine
- [ ] https://github.com/muxable/rtchat/issues/1166
- [ ] https://github.com/muxable/rtchat/issues/1165
- [ ] https://github.com/muxable/rtchat/issues/1183
- [ ] https://github.com/muxable/rtchat/issues/1184
- [ ] https://github.com/muxable/rtchat/issues/1185
- [ ] https://github.com/muxable/rtchat/issues/1187
- [ ] https://github.com/muxable/rtchat/issues/1188
- [ ] https://github.com/muxable/rtchat/issues/1186
This is going to be a project to overhaul how TTS works in RealtimeChat. Today, it works by attempting to keep the main activity alive while the active channel is streaming. This comes with some complications, namely:
This project will have four components:
The Android and iOS TTS engines will have the same Dart APIs following: