jagin / detectron2-pipeline

Modular image processing pipeline using OpenCV and Python generators powered by Detectron2.

Detectron2 pipeline

Modular image processing pipeline using OpenCV and Python generators powered by Detectron2.

For detailed description how to construct image processing pipeline using OpenCV and Python generators read the following Medium stories in order:

• Modular image processing pipeline using OpenCV and Python generators
• Video processing pipeline with OpenCV
• How to embed Detectron2 in your computer vision project
• PoseFlow - real-time pose tracking

Setup environment

This project is using Conda for project environment management.

Setup the project environment:

$ conda env create -f environment.yml
$ conda activate detectron2-pipeline

or update the environment if you git pull the repo previously:

$ conda env update -f environment.yml

Install Detectron2 in a folder one level above the detectron2-pipeline project folder

$ cd ..
$ git clone https://github.com/facebookresearch/detectron2.git
$ cd detectron2
$ git checkout 3def12bdeaacd35c6f7b3b6c0097b7bc31f31ba4
$ python setup.py build develop

We checkout 3def12bdeaacd35c6f7b3b6c0097b7bc31f31ba4 commit to ensure that you can use the code out of the box with this repo.

If you got any problems with Detectron2 installation please refer to INSTALL.md.

Demo

Run the command to execute inferences on images from the selected folder:

$ python process_images.py -i assets/images/friends -p

By default the instance segmentation model is used from configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml. You can try other models with --config-file option, for example:

$ python process_images.py -i assets/images/friends -p --config-file configs/COCO-Keypoints/keypoint_rcnn_R_50_FPN_3x.yaml

For video processing run:

$ python process_video.py -i assets/videos/walk.small.mp4 -p -d -ov walk.small.mp4

Tests

pytest is used as a test framework. All tests are stored in tests folder. Run the tests:

$ pytest

Resources and Credits

• For Unix like pipeline idea credits goes to this Gist
• The source of the example images and videos is pixbay
• Some ideas and code snippets are borrowed from pyimagesearch
• Color constants from Python Color Constants Module
• Detectron2
• Faster video file FPS with cv2.VideoCapture and OpenCV
• Applications of Foreground-Background separation with Semantic Segmentation
• Pose tracking based on PoseFlow: Efficient Online Pose Tracking

License

MIT License