Google Assistant API gets only ask one question

[akvkickstart]

Hi,

Whenever I run google_assistant_for_raspberry_pi.py, I can ask only one question and stops listening.

pi@rpigg:~/mic_array_akv/mic_array $ python google_assistant_for_raspberry_pi.py
ON_MUTED_CHANGED:
{u'is_muted': False}
ON_START_FINISHED

ON_CONVERSATION_TURN_STARTED
ON_END_OF_UTTERANCE
ON_RECOGNIZING_SPEECH_FINISHED:
{u'text': u'what is the square root of 49'}
ON_RESPONDING_STARTED:
{u'is_error_response': False}
ON_RESPONDING_FINISHED
ON_CONVERSATION_TURN_FINISHED:
{u'with_follow_on_turn': False}
^C^C^CTraceback (most recent call last):
File "google_assistant_for_raspberry_pi.py", line 81, in
main()
File "google_assistant_for_raspberry_pi.py", line 77, in main
process_event(event)
File "/usr/local/lib/python2.7/dist-packages/google/assistant/library/assistant.py", line 105, in exit
self._lib.assistant_free(self._inst)
KeyboardInterrupt

The same issue happens when I run google-assistant-demo, it looks like the mic array is not listening anymore.

Can you please help diagnose this issue?

Thanks

[Yihui Xiong]

After you stop google-assistant-demo, will you be able to record audio with the mic array?

[akvkickstart]

Yes, I was able to record audio using the mic array. I also run google_assistant_for_raspberry_pi.py after running google-assistant-demo and I still get one question asked to google.

[Yihui Xiong]

It's wierd!
Maybe it is blocked at playback. Which audio output do you use? The on-board audio output may not work properly sometimes.

[Yihui Xiong]

To verify if it's a problem of playback. Try to play a audio when google-assistant-demo is blocked.

Or use the audio output of the mic array (need to solder the audio jack).

[akvkickstart]

Yes I am using the on-board audio output of the Mic array, can you please tell me how I can test the playback issue? Also I have soldered the audio jack of the Mic array...

[Yihui Xiong]

You can create ~/.asoundrc with the following content to use the mic array's output.

pcm.!default {
    type plug
    slave.pcm "hw:1"
}

[akvkickstart]

Yep, I already created that:

pi@rpigg:~ $ cat .asoundrc
pcm.!default {
type asym
capture.pcm "mic"
playback.pcm "speaker"
}
pcm.mic{
type plug
slave {
pcm "hw:1,0"
}
}
pcm.speaker {
type plug
slave {
pcm "hw:1,0"
}
}

[akvkickstart]

I opened 2 putty terminal, on the first window, I opened google-assistant-demo and the second putty terminal, I tried to record and tried aplay as well:

First Putty window:

pi@rpigg:~ $ google-assistant-demo
ON_MUTED_CHANGED:
{u'is_muted': False}
ON_START_FINISHED

ON_CONVERSATION_TURN_STARTED
ON_END_OF_UTTERANCE
ON_RECOGNIZING_SPEECH_FINISHED:
{u'text': u'what is the time now'}
ON_RESPONDING_STARTED:
{u'is_error_response': False}
ON_RESPONDING_FINISHED
ON_CONVERSATION_TURN_FINISHED:
{u'with_follow_on_turn': False}

Second Putty window:

pi@rpigg:~ $ aplay rectest-01.wav
Playing WAVE 'rectest-01.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo
pi@rpigg:~ $ aplay rectest-01.wav
Playing WAVE 'rectest-01.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo
pi@rpigg:~ $ arecord -D plughw:1,0 --duration=5 -f cd -vv /akvwav.wav
arecord: main:722: audio open error: Device or resource busy
pi@rpigg: $ aplay rectest-01.wav
Playing WAVE 'rectest-01.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo

[akvkickstart]

Any updates on this one? I was thinking of buying another Mic Array but due to this issue I might not buy one.

[Yihui Xiong]

To diagnose it, I think we can enable multiple applications to record from the same device simultaneously. When google assistant is blocked, we use another application (arecord) to record audio and check if the recorded audio is broken.

1. Use alsa dsnoop plugin

2. try to use pulseaudio as a middle layer
   alsa <---> pulseaudio <----> googel assistant

It'd be better use a desktop environment via VNC or HDMI, then we can use audacity and pavucontrol (tool to configure pulseaudio) to diagnose.

[akvkickstart]

I tried to install pulseaudio but it doesn't work if pulseaudio is enabled.

[akvkickstart]

any help guys?

[Yihui Xiong]

No ideas~
Have you tried alsa dsnoop plugin?

[akvkickstart]

can you tell me how I can set that up? Thanks/

[Yihui Xiong]

Here is a configuration with dmix and dsnoop:

# The IPC key of dmix or dsnoop plugin must be unique
# If 555555 or 666666 is used by other processes, use another one

pcm.!default {
    type asym
    playback.pcm "playback"
    capture.pcm "capture"
}

pcm.playback {
    type plug
    slave.pcm "dmixed"
}

pcm.capture {
    type plug
    slave.pcm "array"
}

pcm.dmixed {
    type dmix
    slave.pcm "hw:0,0"
    ipc_key 555555 
}

pcm.array {
    type dsnoop
    slave {
        pcm "hw:0,0"
    }
    ipc_key 666666
}