This platform was built to modularize robotics development and experimentation with Python/C++ using Raspberry Pi/Jetson nano and Arduino.
To use the Googla Coral USB Accelerator, first reprint the Pi SD card with the image included in the AIY Maker Kit.
(I tried to install the required software included in the Coral getting started guide, but was unsuccessful due to an error that "GLIBC_2.29" was not found.)
Alternatively, you can opt for the original (slower) facial recognition process by setting Config.VISION_TECH to opencv. I'm no longer updating this section, so you may encounter some integration issues.
chmod 777 install.sh
./install.sh
To use neopixels on Raspberry Pi, you need to disable the sound (see neopixel section).
./startup.sh
For manual control via keyboard
./manual_startup.sh
Contains a preview of the video feed to get started (not available via SSH)
./preview_startup.sh
python3 -m pytest --cov=modules --cov-report term-missing
Run sudo vim /etc/rc/local and add the following lines before exit 0:
python3 /home/Desktop/companion-robot/shutdown_pi.py
/home/Desktop/companion-robot/startup.sh
GPIO 26 is wired to enable shutdown when brought to ground via a switch.
The shutdown_pi.py script manages this.
Guide: https://howchoo.com/g/mwnlytk3zmm/how-to-add-a-power-button-to-your-raspberry-pi
Using Raspberry Pi camera or any webcam
8 servo control with Arduino serial connection (6 servo feet, 1 servo pan, 1 servo tilt)
It is integrated externally and via Arduino serial connection via software.
A buzzer is connected to GPIO 27 so that tones can be played when no sound is output (see Neopixel section).
https://github.com/gumslone/raspi_buzzer_player.git
An RCWL-0516 microwave radar sensor is attached to GPIO 13. This sensor, with its piservo module, will automatically scan 360 degrees like a radar when it enters a night loop
note: it contains two different piservos, one for nltk and one for motion sensor
NLTK analyzes a text and evaluates the degree to which the text is positive or negative. The antenna again uses the piservo control to perform an animation of this evaluation.
GPIO 18, 19 and 20 are used to use stereo MEMS microphones as audio input.
Mic 3V - Connects to Pi 3.3V.
Mic GND - Connects to Pi GND.
Mic SEL - Pi connects to GND (this is used for channel selection, can be connected to 3.3V or GND).
Mic BCLK - Connects to BCM 18 (pin 12).
Mic DOUT - Connects to BCM 20 (pin 38).
Mic LRCL - connects to BCM 19 (pin 35).
https://learn.adafruit.com/adafruit-i2s-mems-microphone-breakout/raspberry-pi-wiring-test
cd ~
sudo pip3 install --upgrade adafruit-python-shell
wget https://raw.githubusercontent.com/adafruit/Raspberry-Pi-Installer-Scripts/master/i2smic.py
sudo python3 i2smic.py
arecord -l
arecord -D plughw:0 -c2 -r 48000 -f S32_LE -t wav -V stereo -v file_stereo.wav
Note: See additional configuration below to support voice recognition.
The original hot word detection used the now deprecated Snowboy. The files are still available in this repository:
https://github.com/dmt-labs/modular-biped/blob/feature/PR29_review/modules/hotword.py contains the module from this framework.
https://github.com/dmt-labs/modular-biped/tree/feature/PR29_review/modules/snowboy includes original snowboy functionality.
https://github.com/dmt-labs/modular-biped/tree/feature/PR29_review/modules/snowboy/resources/models contains trained models that can be used as keywords. It may be possible to find more on the internet.
A guide to training new models is available here, but there are resources for training new models on a Linux device.
Trigger word for voice recognition (not currently used): https://snowboy.kitt.ai/
Speech recognition is activated when a face is seen.
Make sure that the device_index value specified in the modules/speechinput.py file matches your microphone.
See scripts/speech.py to list and test input devices. See below for MEMS microphone configuration.
By default, the Adafruit I2S MEMS Microphone Breakout does not work with voice recognition.
The following configuration changes must be made to support voice recognition on MEMS microphone(s).
sudo apt-get install ladspa-sdk
Create /etc/asound.conf file with the following content:
pcm.pluglp {
type ladspa
slave.pcm "plughw:0"
path "/usr/lib/ladspa"
capture_plugins [
{
label hpf
id 1042
}
{
label amp_mono
id 1048
input {
controls [ 30 ]
}
}
]
}
pcm.lp {
type plug
slave.pcm pluglp
}
This ensures that the 'lp' device can be referenced in voice recognition. In the example below it is shown at index 18.
For example, the sample rate should be set to 16000.
mic = sr.Microphone(device_index=18, sample_rate=16000)
References:
To use the Raspberry Pi's serial port, getty (the program that displays the login screen) must be disabled.
sudo raspi-config -> Interfacing Options -> Serial -> "Would you like a login shell to be accessible over serial" = 'No'. Restart
Connect GPIO pins 14 and 15 (tx and rx) of the Raspberry Pi to the Arduino tx and rx pins (tx -> rx in both direction and direction reversal!) via a logic level shifter, since Raspberry Pi 3v3 and Arduino (probably) It is 5v.
To upload via serial pins, press the reset button on the Arduino at the point where the IDE starts 'uploading' (after compiling); otherwise a synchronization error will be displayed.
WS1820B support is provided via Pi GPIO pin 12. Unfortunately, audio on the Pi must be disabled to support this.
sudo vim /boot/config.txt
#dtparam=audio=on
Therefore, the application must be run with sudo.
If you want to use neopixels without using the driver (and no audio output), set pin to 12 in the configuration and set i2c to False. The application must also be run with sudo.
https://learn.adafruit.com/neopixels-on-raspberry-pi/python-usage
The live translation module translation.py allows live translation of logs and TTS output via Google Translate.
Call the translate.request(msg) function and optionally specify the source and target languages.
config/translation.yml specifies the default languages.
The prefabricated PCBs of this project are available in the circuits folder. This provides connectivity between core components as described above.
