optimal16 / Tensorflow-TensorRT

This repository is for my YT video series about optimizing a Tensorflow deep learning model using TensorRT. We demonstrate optimizing LeNet-like model and YOLOv3 model, and get 3.7x and 1.5x faster for the former and the latter, respectively, compared to the original models.

This repository is for my YouTube video series here, about optimizing deep learning model using TensorRT. We demonstrate optimizing LeNet-like model and YOLOv3 model, and get 3.7x and 1.5x faster for the former and the latter, respectively, compared to the original models. For the details and how to run the code, see the video below.

List of Videos

1. Optimizing Tensorflow Model to TensorRT
2. Visualizing Before and After TensorRT Optimization
3. Optimizing Keras Model to TensorRT
4. Train Keras Model Using Your Own Dataset
5. Restore and Use Stored Keras Model to Perform Inference
6. Optimizing YOLOv3 using TensorRT
7. Another YOLOv3 Detection Result (Native Tensorflow vs TensorRT optimized)

Pre-requirement

1. TensorRT: follow the tutorial here for Ubuntu dekstop or here for Jetson devices, to install tensorRT

Library I Use:

1. Tensorflow 1.12
2. OpenCV 3.4.5
3. Pillow 5.2.0
4. Numpy 1.15.2
5. Matplotlib 3.0.0

Environment used in this video series

Dekstop PC

1. OS: Ubuntu 16.04 - 64bit
2. GPU: GeForce 1060 6Gb
3. Driver version: 384.130
4. RAM: 16Gb
5. CUDA: 9.0
6. CuDNN: 7
7. TensorRT: 4.1.2

Jetson TX2

1. OS: Ubuntu 16.04 - 64bit
2. GPU (with RAM shared): 8Gb
3. Driver version: Given along flashing with Jetpack 3.3
4. CUDA: 9.0
5. CuDNN: 7
6. TensorRT: 4.1.3

Dataset

Download (subset of) MNIST dataset here, extract and put in folder dataset.

YOLOv3 Frozen Model

Download here, extract and put in folder model/YOLOv3