shining-love / tensorrt-alpha

TensorRT-Alpha implements CUDA accelerated deployment of more than 30 models, including but not limited to yolov8, yolov7, yolov6, yolov5, yolov4, yolov3, yolox, yolor based on subsequent Vision-Tranformer related deep learning models

TensorRT-Alpha

English | 简体中文

Introduce

This repository provides accelerated deployment cases of deep learning CV popular models, and cuda accelerated methods for pre-processing and post-processing of mainstream models. Most of the model transformation process is torch->onnx->tensorrt. There are two ways to obtain onnx files:

• According to the network disk provided by this repository, download ONNX directly
• Follow the instructions provided in this repository to manually export ONNX from the relevant source code framework.

graph LR
    pytorch/tensorflow -->onnx-->tensorrt

Update

• 2023.01.01 🔥 update yolov3, yolov4, yolov5, yolov6
• 2023.01.04 🍅 update yolov7, yolox, yolor
• 2023.01.05 🎉 update u2net, libfacedetction
• 2023.01.08 🚀 The whole network is the first to support yolov8

Installation

Compatible platforms: windows、linux

• cuda11.6
• cudnn8.4
• tensorrt8.4.2.4
• opencv3.x

python dependent environment(optional）：

# install miniconda first
conda create -n tensorrt-alpha python==3.8 -y
conda activate tensorrt-alpha
git clone https://github.com/FeiYull/tensorrt-alpha
cd tensorrt-alpha
pip install -r requirements.txt  

Run

linux

set your TensorRT_ROOT path:

cd tensorrt-alpha/cmake
vim common.cmake
# set var TensorRT_ROOT to your path in line 20, eg:
# set(TensorRT_ROOT /root/TensorRT-8.4.2.4)

start to build project: For example:yolov7

windows

waiting for update

Onnx

At present, more than 30 models have been implemented, and some onnx files of them are organized as follows:

model	weiyun	google driver
yolov3	weiyun	google driver
yolov4	weiyun	google driver
yolov5	weiyun	google driver
yolov6	weiyun	google driver
yolov7	weiyun	google driver
yolov8	weiyun	google driver
yolox	weiyun	google driver
yolor	weiyun	google driver
u2net	weiyun	google driver
libfacedet	weiyun	google driver
facemesh	weiyun	google driver
more...(🚀: I will be back soon!)

Visualization

some precision alignment renderings comparison:

yolov8n : Offical( left ) vs Ours( right )


yolov7-tiny : Offical( left ) vs Ours( right )


yolov5s : Offical( left ) vs Ours( right )


libfacedetction : Offical( left ) vs Ours( right topK:4000)

Reference

[0].https://github.com/NVIDIA/TensorRT
[1].https://github.com/onnx/onnx-tensorrt
[2].https://github.com/NVIDIA-AI-IOT/torch2trt
[3].https://github.com/shouxieai/tensorRT_Pro
[4].https://github.com/opencv/opencv_zoo