Computer Vision on Raspberry Pi

by Danh Doan

Introduction

This is a series about developing common Computer Vision projects on Raspberry Pi board. Some of them requires the support of Movidius Neural Compute Stick to boost the performance. OpenVINO toolkit is mainly the development tool that helps optimize the hardware and models to work well with Raspi

The main purpose of this work is to help developers from all levels to gain insights and resources to working with Raspberry Pi for Computer Vision projects. Every sample project is refactored and organized so that it is easily understandable and approachable.

If you have any project ideas and issues with those projects, feel free to comment. It will help improve and enrich the contents. Thanks in advance with my sincere.

Other Computer Vision demos: [link]

Updates

2019, Nov 27:

Add 012-tflite-object-detection

2019, Nov 15:

Add openvino-models: [link]

2019, Nov 05:

Add 008-pi-emotion-recognition
Update installation guide to support IECore

2019, Oct 23:

Add 004-pi-head-pose-estimation

2019, Oct 16:

Add 006-pi-face-verification

2019, Oct 15:

Add 003-pi-face-alignment

2019, Oct 12:

Add 005-pi-object-detection
Add 002-pi-facial-landmark-detection
Add 001-pi-face-detection
Add 000-show-pi-camera

Sample Projects

Test with Pi camera module: [link]
- Play around with builtin Pi camera module
Face Detection with High Accuracy: [link]
- Develop an accurate and robust Face detector with pretrained SSD model trained from WIDER dataset
Facial Keypoint Detection: [link] [demo] [demo]
- Develop a simple Facial keypoints localization that detect 5 main keypoints of human faces (center eyes, nose tip, and mouth corner
Face Alignment: [link]
- Based on 5 keypoints, align human faces, to support other problem e.g. Face Identification, Face Verification, ...
Headpose Estimation: [link] [demo]
- Estimate Human head pose in Tait-Bryan angles (yaw, pitсh or roll)
Human Detection: [link] [demo]
- Develop an Object detector especially with SSD model
Notice: human is just an example of objects, any object detection model can be converted to work with this sample project
Face Verification: [link]

Verify Face Identity by Face embeddings
Emotion Recognition: [link] [demo]

Recognize Emotional states of Human faces
Car and License Plate Detection: [ongoing]
TFLite Object Detection: [link] [demo]

Installation

Follow install.md instructions [link] to install essential packages and modules for working with Raspberry Pi. Installing OpenVINO as a toolkit to the development.

Usage

Clone this repository:

cd ~

mkdir workspace && cd workspace

git clone https://github.com/danhdoan/computer-vision-raspberrypi
Download OpenVINO pretrained-model

mkdir openvino-models tflite-models

You can notice a soft symbol link in any projects that maps to this directory. If you want to store it elsewhere, beware of re-map this symbol link. To download a model, just go to the official OpenVINO site from Intel:

https://download.01.org/opencv/2019/open_model_zoo/R1/

In this project, models from R1 sub-dir are used. R3 is currently the latest, it also works well with sample code. Just download the FP16 models, they can be applied to Raspberry Pi. In my code, I usually add a postfix -fp16 to clarify this issue. Thus, download a pretrained model and change its name correspondingly.

Run a sample project

All sample projects have the same argument parser

 usage: <app-name>.py [-h] [--video VIDEO] [--name NAME] [--show]                              
                           [--record] [--flip_hor] [--flip_ver]
 optional arguments:                                                                                    
   -h, --help            show this help message and exit
   --video VIDEO, -v VIDEO
                         Video Streamming link or Path to video source
   --name NAME, -n NAME  Name of video source
   --show, -s            Whether to show the output visualization
   --record, -r          Whether to save the output visualization
   --flip_hor, -fh       horizontally flip video frame
   --flip_ver, -fv       vertically flip video frame

It is implemented in altusi/helper.py, you can customize as you wanted. To see argument list of a sample, e.g. object-detection, just type:

 python3 app-object-detector.py -h

References

OpenVINO model zoo: https://download.01.org/opencv/2019/open_model_zoo/R3/
Official Intel site:
- https://docs.openvinotoolkit.org/latest/_demos_README.html
- https://docs.openvinotoolkit.org/latest/_models_intel_index.html
Raspberry Pi and OpenVINO installation: https://www.hackster.io/news/getting-started-with-the-intel-neural-compute-stick-2-and-the-raspberry-pi-6904ccfe963

jiakang / computer-vision-raspberrypi