Voice chat support
psugihara opened this issue · comments
There's a fantastic demo here https://github.com/ggerganov/whisper.cpp/tree/master/examples/talk-llama. I think a good UI could be putting a button in the text input like iOS messages. Hitting that would put you into voice mode. Apple's built in TTS is probably fine for now for the response (though I wonder if it has streaming support)
Step 1 would be rigging up a little prototype/demo like georgi's linked above but running in FreeChat.
could be useful
https://github.com/yl4579/StyleTTS2
"Apple's built in TTS is probably fine for now"
On Apple Silicon, could maybe use Apple's on-device speech recognition to keep the file size down... https://developer.apple.com/documentation/speech