How to deploy ONNX models using DeepStream on Jetson Nano. [Blog] [Performance]
This repository provides complementary material to this blog post about deploying an ONNX object detection model using the DeepStream SDK on Jetson Nano. Various experiments were designed to test the features and performance of DeepStream.
- Follow the instructions in the blog to setup your Jetson Nano and to install the DeepStream SDK.
- Use the bellow commands to clone and move into the repository.
git clone https://github.com/thatbrguy/Deep-Stream-ONNX.git
cd Deep-Stream-ONNX
- Download the Tiny YOLOv2 ONNX model from the ONNX Model Zoo. We used this model in our experiments.
- A custom bounding box parser function is written in
nvdsparsebbox_tiny_yolo.cpp
inside thecustom_bbox_parser
folder. - A
Makefile
is configured to compile the custom bounding box parsing function into a shared library (.so) file. It is also available inside the same folder. - The below variables may need to be set by the user in the
Makefile
before compiling:
# Set the CUDA version.
CUDA_VER:=10
# Name of the file with the custom bounding box parser function.
SRCFILES:=nvdsparsebbox_tiny_yolo.cpp
# Name of the shared library file to be created after compilation.
TARGET_LIB:=libnvdsinfer_custom_bbox_tiny_yolo.so
# Path to the DeepStream SDK. REPLACE /path/to with the location in your Jetson Nano.
DEEPSTREAM_PATH:=/path/to/deepstream_sdk_v4.0_jetson
Note: If no changes were made to the code by the user, and the blog was followed to set up Jetson Nano and DeepStream, then only the DEEPSTREAM_PATH variable may need to be set before compilation. Default values can be used for the other three variables.
- Once the variables are set, save the
Makefile
. Compile the custom bounding box parsing function using:make -C custom_bbox_parser
.
- Download the
sample.tar.gz
from this drive link. Extract thevids
directory into theDeep-Stream-ONNX
directory. - You can launch DeepStream using the following command:
deepstream-app -c ./config/deepstream_app_custom_yolo.txt
- You can edit the config files inside the
config
to alter various settings. You can refer to the blog for resources on understanding the various properties inside the config files.
- Methods for quickly verifying if an ONNX model will be accepted by DeepStream (v4.0):