onnx file won't work in deepstream_python_apps without EXPLICIT_BATCH flag set
whutchi opened this issue · comments
I have a trained detection model that I trained in this excellent Jetson-Inference series. In that process instructed by pytorch-ssd.md page, training produces a .pth file that is then converted by the onnx_export.py utility to an onnx model. I want to use that trained model in one of the deepstream_python_apps that are perfect for my needs. To use an onnx model, you change the config file to comment out the tlt-encoded-model and the tlt-model-key, add a line with the name of the onnx file, then use trtexec to create an engine file from the onnx file and put that in the model-engine-file entry in place of the sample that was there. With that setup, it starts up great, but then hits an error saying that the onnx model must have been created with NetworkDefinitionCreationFlag.EXPLICIT_BATCH, since trt no longer accepts implict batch size. Adding the argument --explictBatch to the trtexec command doesn't solve it. The documentation at https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#explicit-implicit-batch also doesn't help.
I've tried to get around that by going all the way back to the training and the conversion to onnx and setting the batch size explicitly, but that does not fulfill the requirement. Is there any way to use an onnx network created by the training or the onnx_export.py conversion to be able to run in the deepstream_python_apps? Perhaps some way something could be added to the onnx_export code to set that parameter?
I've got such a nice trained onnx network and the deepstream_python_app examples are perfect for what I want to do if I can solve this.
It does seem like a major flaw to have the main network training method in the dusty-nv/jetson-inference tutorials produce an onnx file that can't be used in the main sample apps in the deepstream_python_apps.... How can that be remedied?
