[Bug] RT-DETR failed in TVM with grid_sampler
xdd12135 opened this issue · comments
Checklist
- I have searched related issues but cannot get the expected help.
- 2. I have read the FAQ documentation but cannot get the expected help.
- 3. The bug has not been fixed in the latest version.
Describe the bug
When I use the onnx I export in TVM, I got this error:
Traceback (most recent call last): File "dump_relay_ir.py", line 59, in <module> save_relay_ir(args.path, args.relay_ir_file, args.dtype, pass_path=args.pass_path, toposort=args.toposort) File "dump_relay_ir.py", line 38, in save_relay_ir ir_module, param = relay.frontend.from_onnx(onnx_model, dtype=dtype) File "/data/application/xxx/anaconda3/envs/xxx/lib/python3.8/site-packages/tvm/relay/frontend/onnx.py", line 6246, in from_onnx mod, params = g.from_onnx(graph, opset) File "/data/application/xxx/anaconda3/envs/xxx/lib/python3.8/site-packages/tvm/relay/frontend/onnx.py", line 5911, in from_onnx self._check_for_unsupported_ops(graph) File "/data/application/xxx/anaconda3/envs/xxx/lib/python3.8/site-packages/tvm/relay/frontend/onnx.py", line 5998, in _check_for_unsupported_ops raise tvm.error.OpNotImplemented(msg) tvm.error.OpNotImplemented: The following operators are not supported for frontend ONNX: grid_sampler
but I found a operators called GridSample , this two suppose to be the same. I want to know why the onnx model I export have the different operator name of grid_sampler.
Reproduction
Here is my export code
python tools/torch2onnx.py mmdeploy/configs/mmdet/detection/detection_onnxruntime_static.py /mmdetection-main/projects/CO-DETR/configs/codino/co_dino_5scale_swin_l_16xb1_16e_o365tococo.py co_dino_5scale_swin_large_16e_o365tococo-614254c9.pth mmdetection-main/demo/demo.jpg --work-dir swin_20 --device cuda
And here is my result:
`
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
/data/application/xxx/anaconda3/envs/torch201/lib/python3.8/site-packages/torch/onnx/utils.py:689: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at ../torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
_C._jit_pass_onnx_graph_shape_type_inference(
/data/application/xxx/anaconda3/envs/torch201/lib/python3.8/site-packages/torch/onnx/utils.py:1186: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at ../torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
_C._jit_pass_onnx_graph_shape_type_inference(
03/25 17:58:14 - mmengine - INFO - Execute onnx optimize passes.
03/25 17:58:14 - mmengine - WARNING - Can not optimize model, please build torchscipt extension.
More details: https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/experimental/onnx_optimizer.md
============= Diagnostic Run torch.onnx.export version 2.0.1+cu117 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 60 WARNING 0 ERROR =======================
60 WARNING were not printed due to the log level.
03/25 17:58:19 - mmengine - INFO - torch2onnx finished. Results saved to swin_20
`
Is this bug related to the printed Warning : [W shape_type_inference.cpp:1920] Warning: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
Environment
Here is my environment :
mmcv 2.1.0
mmdeploy 1.3.1
mmdet 3.3.0
mmengine 0.10.3
torch 2.0.1Error traceback
No response