TensorRT doesn't provide the same output as Torch model

[Phan Nguyen Duc Hieu]

Before Asking

• I have read the README carefully. 我已经仔细阅读了README上的操作指引。

• I want to train my custom dataset, and I have read the tutorials for finetune on your data carefully and organize my dataset correctly; 我想训练自定义数据集，我已经仔细阅读了训练自定义数据的教程，以及按照正确的目录结构存放数据集。

• I have pulled the latest code of main branch to run again and the problem still existed. 我已经拉取了主分支上最新的代码，重新运行之后，问题仍不能解决。

Search before asking

• I have searched the DAMO-YOLO issues and found no similar questions.

Question

I have converted the torch model to tensorRT model using end2end converter.
python tools/converter.py -f configs/damoyolo_tinynasL45_L.py -c best.pth --batch_size 1 --img_size 1024 --trt --end2end --trt_eval

This command worked normally but the evaluation is 0% , compare to 90% when i using torch model
I also use demo command to predict some images. The output show that the bounding box seem randomly.

I also try convert to onnx by this command:
python tools/converter.py -f configs/damoyolo_tinynasL45_L.py -c best.pth --batch_size 1 --img_size 1024
The Onnx model output exactly the same output as torch model

So do i miss any configuration for tensorRT?

Additional

No response

[Yiqi Jiang]

Hello, currently the End2End NMS module is only compatible with TensorRT-7.2.1.4. Please verify if the TensorRT version you are using is consistent. If it is not, you may consider modifying the TensorRT version or exporting it using a non-End2End TensorRT and implementing NMS post-processing using Python.

[Adam J Tegen]

I have run into this issue as well. The issue is that the TRT8 export has passed the wrong value for box_coding to TRT::EfficientNMS_TRT. I've confirmed that PR #113 fixes the issue. It changes box_coding from 1 (BoxCenterSize) to 0 (BoxCorner).
See https://github.com/NVIDIA/TensorRT/tree/release/8.6/plugin/efficientNMSPlugin for more information on the parameters.