Magellen / jetson-inference

Guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

Deploying Deep Learning

Welcome to our training guide for inference and deep vision runtime library for NVIDIA DIGITS and Jetson Xavier/TX1/TX2.

This repo uses NVIDIA TensorRT for efficiently deploying neural networks onto the embedded platform, improving performance and power efficiency using graph optimizations, kernel fusion, and half-precision FP16 on the Jetson.

Vision primitives, such as imageNet for image recognition, detectNet for object localization, and segNet for semantic segmentation, inherit from the shared tensorNet object. Examples are provided for streaming from live camera feed and processing images from disk. See the Deep Vision API Reference Specification for accompanying documentation.

There are multiple tracks of the tutorial that you can choose to follow, including Training + Inference or Inference-Only.

Hello AI World (Inference Only)

If you would like to only do the inference portion of the tutorial, which can be run on your Jetson in roughly two hours, these modules are available below:

• Setting up Jetson with JetPack
• Building the Repo from Source
• Classifying Images with ImageNet
  • Using the Console Program on Jetson
  • Coding Your Own Image Recognition Program
  • Running the Live Camera Recognition Demo
• Locating Object Coordinates using DetectNet
  • Detecting Objects from the Command Line
  • Running the Live Camera Detection Demo

Two Days to a Demo (Training + Inference)

The full tutorial includes training and inference, and can take roughly two days or more depending on system setup, downloading the datasets, and the training speed of your GPU.

• DIGITS Workflow
• DIGITS System Setup
• Setting up Jetson with JetPack
• Building the Repo from Source
• Classifying Images with ImageNet
  • Using the Console Program on Jetson
  • Coding Your Own Image Recognition Program
  • Running the Live Camera Recognition Demo
  • Re-Training the Network with DIGITS
  • Downloading Image Recognition Dataset
  • Customizing the Object Classes
  • Importing Classification Dataset into DIGITS
  • Creating Image Classification Model with DIGITS
  • Testing Classification Model in DIGITS
  • Downloading Model Snapshot to Jetson
  • Loading Custom Models on Jetson
• Locating Object Coordinates using DetectNet
  • Detection Data Formatting in DIGITS
  • Downloading the Detection Dataset
  • Importing the Detection Dataset into DIGITS
  • Creating DetectNet Model with DIGITS
  • Testing DetectNet Model Inference in DIGITS
  • Downloading the Detection Model to Jetson
  • DetectNet Patches for TensorRT
  • Detecting Objects from the Command Line
  • Multi-class Object Detection Models
  • Running the Live Camera Detection Demo on Jetson
• Semantic Segmentation with SegNet
  • Downloading Aerial Drone Dataset
  • Importing the Aerial Dataset into DIGITS
  • Generating Pretrained FCN-Alexnet
  • Training FCN-Alexnet with DIGITS
  • Testing Inference Model in DIGITS
  • FCN-Alexnet Patches for TensorRT
  • Running Segmentation Models on Jetson

Extra Resources

In this area, links and resources for deep learning developers are listed:

• Appendix
  • ros_deep_learning - TensorRT inference ROS nodes
  • NVIDIA AI IoT - NVIDIA Jetson GitHub repositories
  • Jetson eLinux Wiki - Jetson eLinux Wiki

Recommended System Requirements

Training GPU: Maxwell, Pascal, Volta, or Turing-based GPU (ideally with at least 6GB video memory)
                        optionally, AWS P2/P3 instance or Microsoft Azure N-series
                        Ubuntu 14.04 x86_64 or Ubuntu 16.04 x86_64.

Deployment:   Jetson Xavier Developer Kit with JetPack 4.0 or newer (Ubuntu 18.04 aarch64).
                        Jetson TX2 Developer Kit with JetPack 3.0 or newer (Ubuntu 16.04 aarch64).
                        Jetson TX1 Developer Kit with JetPack 2.3 or newer (Ubuntu 16.04 aarch64).

note: this branch is verified against the following BSP versions for Jetson AGX Xavier and Jetson TX1/TX2:
             > Jetson AGX Xavier - JetPack 4.1.1 DP / L4T R31.1 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0 GA
             > Jetson AGX Xavier - JetPack 4.1 DP EA / L4T R31.0.2 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0 RC
             > Jetson AGX Xavier - JetPack 4.0 DP EA / L4T R31.0.1 aarch64 (Ubuntu 18.04 LTS) inc. TensorRT 5.0 RC
             > Jetson TX2 - JetPack 3.3 / L4T R28.2.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 4.0
             > Jetson TX1 - JetPack 3.3 / L4T R28.2 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 4.0
             > Jetson TX2 - JetPack 3.2 / L4T R28.2 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 3.0
             > Jetson TX2 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 3.0 RC
             > Jetson TX1 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 3.0 RC
             > Jetson TX2 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 2.1
             > Jetson TX1 - JetPack 3.1 / L4T R28.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 2.1
             > Jetson TX2 - JetPack 3.0 / L4T R27.1 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 1.0
             > Jetson TX1 - JetPack 2.3 / L4T R24.2 aarch64 (Ubuntu 16.04 LTS) inc. TensorRT 1.0
             > Jetson TX1 - JetPack 2.3.1 / L4T R24.2.1 aarch64 (Ubuntu 16.04 LTS)

Note that TensorRT samples from the repo are intended for deployment onboard Jetson, however when cuDNN and TensorRT have been installed on the host side, the TensorRT samples in the repo can be compiled for PC.

Legacy Links

Since the documentation has been re-organized, below are links mapping the previous content to the new locations.

       (click on the arrow above to hide this section)

DIGITS Workflow

See DIGITS Workflow

System Setup

See DIGITS Setup

Running JetPack on the Host

See JetPack Setup

Installing Ubuntu on the Host

See DIGITS Setup

Setting up host training PC with NGC container

See DIGITS Setup

Installing the NVIDIA driver

See DIGITS Setup

Installing Docker

See DIGITS Setup

NGC Sign-up

See DIGITS Setup

Setting up data and job directories

See DIGITS Setup

Starting DIGITS container

See DIGITS Setup

Natively setting up DIGITS on the Host

See DIGITS Native Setup

Installing NVIDIA Driver on the Host

See DIGITS Native Setup

Installing cuDNN on the Host

See DIGITS Native Setup

Installing NVcaffe on the Host

See DIGITS Native Setup

Installing DIGITS on the Host

See DIGITS Native Setup

Starting the DIGITS Server

See DIGITS Native Setup

Building from Source on Jetson

See Building the Repo from Source

Cloning the Repo

See Building the Repo from Source

Configuring with CMake

See Building the Repo from Source

Compiling the Project

See Building the Repo from Source

Digging Into the Code

See Building the Repo from Source

Classifying Images with ImageNet

See Classifying Images with ImageNet

Using the Console Program on Jetson

See Classifying Images with ImageNet

Running the Live Camera Recognition Demo

See Running the Live Camera Recognition Demo

Re-training the Network with DIGITS

See Re-Training the Recognition Network

Downloading Image Recognition Dataset

See Re-Training the Recognition Network

Customizing the Object Classes

See Re-Training the Recognition Network

Importing Classification Dataset into DIGITS

See Re-Training the Recognition Network

Creating Image Classification Model with DIGITS

See Re-Training the Recognition Network

Testing Classification Model in DIGITS

See Re-Training the Recognition Network

Downloading Model Snapshot to Jetson

See Downloading Model Snapshots to Jetson

Loading Custom Models on Jetson

See Loading Custom Models on Jetson

Locating Object Coordinates using DetectNet

See Locating Object Coordinates using DetectNet

Detection Data Formatting in DIGITS

See Locating Object Coordinates using DetectNet

Downloading the Detection Dataset

See Locating Object Coordinates using DetectNet

Importing the Detection Dataset into DIGITS

See Locating Object Coordinates using DetectNet

Creating DetectNet Model with DIGITS

See Locating Object Coordinates using DetectNet

Selecting DetectNet Batch Size

See Locating Object Coordinates using DetectNet

Specifying the DetectNet Prototxt

See Locating Object Coordinates using DetectNet

Training the Model with Pretrained Googlenet

See Locating Object Coordinates using DetectNet

Testing DetectNet Model Inference in DIGITS

See Locating Object Coordinates using DetectNet

Downloading the Model Snapshot to Jetson

See Downloading the Detection Model to Jetson

DetectNet Patches for TensorRT

See Downloading the Detection Model to Jetson

Processing Images from the Command Line on Jetson

See Detecting Objects from the Command Line

Launching With a Pretrained Model

See Detecting Objects from the Command Line

Pretrained DetectNet Models Available

See Detecting Objects from the Command Line

Running Other MS-COCO Models on Jetson

See Detecting Objects from the Command Line

Running Pedestrian Models on Jetson

See Detecting Objects from the Command Line

Multi-class Object Detection Models

See Detecting Objects from the Command Line

Running the Live Camera Detection Demo on Jetson

See Running the Live Camera Detection Demo

Image Segmentation with SegNet

See Semantic Segmentation with SegNet

Downloading Aerial Drone Dataset

See Semantic Segmentation with SegNet

Importing the Aerial Dataset into DIGITS

See Semantic Segmentation with SegNet

Generating Pretrained FCN-Alexnet

See Generating Pretrained FCN-Alexnet

Training FCN-Alexnet with DIGITS

See Training FCN-Alexnet with DIGITS

Testing Inference Model in DIGITS

See Training FCN-Alexnet with DIGITS

FCN-Alexnet Patches for TensorRT

See FCN-Alexnet Patches for TensorRT

Running Segmentation Models on Jetson

See Running Segmentation Models on Jetson

^{© 2016-2019 NVIDIA | Table of Contents}