Realtime Object Detection based on Tensorflow's Object Detection API and DeepLab Project
Release Note: use v1.0 for the original repo that was focused on high performance inference of
ssd_mobilenet
(x10 Performance Increase on Nvidia Jetson TX2)
Release Note: use Master or v2.0 to be additionally able to run and test Mask-Detection Models, KCF-Tracking and DeepLab Models (merge of this project)
The Idea was to create a scaleable realtime-capable object detection pipeline that runs on various systems.
Plug and play, ready to use without deep previous knowledge.
The project includes following work:
- optional download of tensorflow pretrained models
- do Inference with OpenCV, either through video input or on selected test_images.
supported Models are allresearch/object_detection
as well asresearch/deeplab
models - enjoy this project's own
ssd_mobilenet
speed hack, which splits the model in a mutlithreaded cpu and gpu session.
Results in up to x10 performance increase depending on the running system
⇒ which makes it (one of) the fastest inference piplines out there - run statistic tests on sets of images and get statistical information like mean and median fps, std dev and much more
- create
timeline
files measuring the exact time consumption of each operation in your model - inspect, summarize, quantize, transform and benchmark models with the provided
scripts/
- Change Inference Configurations inside
rod/config.py
according to your needs
For example: If you are not interested in visualization: setVISUALIZE
toFalse
,
or if you want to switch off the speed hack setSPLIT_MODEL
toFalse
, - to be able to use KCF_Tracking inside
scripts/
runbash build_kcf.sh
to build it and setUSE_TRACKER
toTrue
to use it
(currently only works for pure object detection models withoutSPLIT_MODEL
) - for realtime inference using video stream run:
python run_objectdetection.py
orpython run_deeplab.py
- for benchmark tests on sample images run:
python test_objectdetection.py
orpython test_deeplab.py
(put them as.jpg
intotest_images/
. timeline results will appear intest_results/
) - Enjoy!
To make use of the tools provided inside scripts/
follow this guide:
- first change all paths and variables inside
config_tools.sh
to your needs / according to your system - When using the first time run:
source config_tools.sh
and in the same terminal run only oncesource build_tools.sh
to build the tools. this will take a while. - For all following uses first run:
source build_tools.sh
(due to the exported variables) and after that you are able to run the wanted scripts always from the same terminal withsource script.sh
. - All scripts log the terminal output to
test_results/
Use the following setup for best and verified performance
- Ubuntu 16.04
- Python 2.7
- Tensorflow 1.4 (this repo provides pre-build tf wheel files for jetson tx2)
- OpenCV 3.3.1
Note: tensorflow v1.7.0 seems to have massive performance issues (try to use other versions)
- Dell XPS 15 with i7 @ 2.80GHZ x8 and GeForce GTX 1050 4GB: 78fps | 105fps
- Nvidia Jetson Tx2 with Tegra 8GB: 30fps | 33 fps
If you like the project, got improvement or constructive critisism, please feel free to open an Issue.
I am always happy to get feedback or help to be able to further improve the project.
Future implementation plans are:
- Add KCF Tracking to improve fps especially on the jetson
-
Mask-SSD: Modify SSD to be able to predict a segmentation mask in parallel to the bounding box - Train a
mask_rcnn Model
with Mobilenet V1/V2 as backbone and deploy it on the Jetson - Split Model and Threading for R-CNN Models
- test_models: A repo for models i am currently working on for benchmark tests
- deeptraining_hands: A repo for setting up the ego- and oxford hands-datasets.
It also contains several scripts to convert various annotation formats to be able to train Networks on different deep learning frameworks
currently supports.xml
,.mat
,.csv
,.record
,.txt
annotations - yolo_for_tf_od_api: A repo to be able to include Yolo V2 in tf's object detection api
- realtime_segmenation: This repo was merged into v2.0
- Mobile_Mask_RCNN: a Keras Model for training Mask R-CNN for mobile deployment
- tf_training: Train Mobile Mask R-CNN Models on AWS Cloud
- tf_models: My
tensorflow/models
fork which includesyolov2
andmask_rcnn_mobilenet_v1_coco