Different SparseConvolution results between C++ and pytorch

[Vincentdaniele]

Hi @hopef thanks for your great work here.

Some background
I am currently exporting mmdetection3d's CenterPoint model (model trained on nuScenes) to run on C++ and TensorRT using CUDA-CenterPoint.
I successfully changed onnx export scripts to adapt mmdetection3d's weights, sparse shape and model layers to tianweiy's CenterPoint, which contains the same layers arranged differently. The pytorch model with fused layers has been tested just before the onnx conversion and give correct outputs compared with the original model.

Main issue
However while testing on the C++ code I have noticeable differences in the ouputs of the sparse convolution model (leading to almost no detection of pedestrians), configs has been changed to support my exported model from mmdetection3d and are exactly the same.
My main guess would be a spconv version issue (as I used 2.3.3 for onnx generation) or a major difference in the onnx / C++ export of the library but I am not able to generate different libspconv has you did for the libspconv-1.1.2 newly provided.

Onnx generation Environment
sys.platform: linux
Python: 3.8.10 (default, May 26 2023, 14:05:08) [GCC 9.4.0]
TorchVision: 0.14.1+cu117
GPU 0: NVIDIA GeForce GTX 1060
NVCC: Cuda compilation tools, release 11.4, V11.4.48
GCC: x86_64-linux-gnu-gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
PyTorch: 1.13.1+cu117
TorchVision: 0.14.1+cu117
OpenCV: 4.2.0
mmdet3d: 1.2.0
Used mmdetection3d "dev-1.x" branch
spconv-cu117 : 2.3.3
onnx : 1.14.1

Running Environment
Nvidia Orin NX 16GB Developer Kit - Jetpack 5.1 [L4T 35.2.1]
Ubuntu 20.04 focal, Release 5.10.104-tegra
cuda-nvcc-11-4/stable,now 11.4.315-1 arm64
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
CUDA : 11.4.315
cuDNN : 8.6.0.166
TensorRT : 5.1

Thank you for any help or hints to help me solve my issue