AlexxIT / StreamAssist

Home Assistant custom component that allows you to turn almost any camera and almost any speaker into a local voice assistant

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add visual custom responses with gifs and tts

relust opened this issue · comments

A while ago I managed to get custom visual responses to play on an android tablet using an Esp32 satellite.
This is a small demo.
Thanks to the fact that AlexxIT managed to introduce wake word detection in the Stream Asssit integration, I thought of replacing the Esp32 satellite that I was using in this project with the Stream Assist integration and Rtpmic app on android tablet.
I added to "Stt start media" custom tts responses and two gifs for speech and listen to be played with Browser Mod popup.
The advantage is that the custom responses are very easy to adapt for each language using your favorite tts service.
Because I made several changes in the Stream Assist integration code, before making the pull request I chose to copy AlexxIT's repository and add it with the changes made to my github page along with the necessary instructions.
@AlexxIT please, if you have time, take a look at this modified repository and tell me your opinion, if you want to add these changes to your integration or you prefer to keep these changes only in the repository modified by me on my github.

And do your changes still need that update? #11

With Rtpmic app on android tablet the delay on stt start media works great by self without that code modiffications. I thinq that issue #11 can be removed.

I want to make random custom responses with your favorite tts service, language and voice.
If I make a service in the Home assistant to send a random tts message in this way, it works very well and randomly plays one answer from the list

service: media_player.play_media
data:
  entity_id: media_player.ha_display2_browser
  media_content_id: media-source://tts/edge_tts?message={{["how can I help you","yes, im listening","how can I assist you"] | random}}&language=en-US-MichelleNeural
  media_content_type: audio

But if I put this media-source://tts/edge_tts?message={{["how can I help you","yes, im listening","how can I assist you"] | random}}&language=en-US-MichelleNeural in the "Stt start media" field in the integration interface, it says all the answers at once and also says random at the end.
@AlexxIT can you tell me how I could do to make the random answers work in Stream Assist integration in this way? I think that, in the integration code, the formatting mode is changed somewhere and it no longer recognizes it as a template. Because at the end it is also played through the home assistant media_player.play_media service and it should work exactly like a home assistant service.

I think you can achieve same result via automations. And triggers to the entitites state change.
Also I can add events to this component.

In VisualStreamAssist integration, gifs are played from the ”local folder” (/config/www/). If someone wants to install this integration they should download the gifs separately and put them in /config/www/ folder. I would like this process to be done automatically when the integration is installed.
@AlexxIT do you know any way to automatically download the gifs needed in the visuals of this integration?

Anyway user should do some additional settings - install browser_mod and more. He can download this files from github with some instruction.