Cannot deploy model to other inference engine (e.g. ONNX, OpenVINO)

[RiverLight4]

Hello,
I'm interested in the RefineMask method and I'd like to use it to cut out the detected image from pictures.
I could train with this repository, but unfortunately, I cannot deploy the trained model to other inference engine. I'm afraid that it is because this official implementation is based on too old MMDetection code (v2.3.0).

I tried with tools/pytorch2onnx.py but the result is failed.
I also tried with MMdeploy v0.14.0 but the result is failed too.
At last, I tried to implement RefineMask into MMDetection v2.28.2 (latest version of 2.x) and tried with MMdeploy v0.14.0. I think I could implement correctly, and it works on MMDetection v2.28.2, but converting the model is failed.

All of them can inference on the MMDetection, but it is failed when converting model with MMDeploy, at torch.jit.trace and torch.jit.script.
I tried with Python 3.7, PyTorch 1.13.1 and CUDA 11.7.

Are there any solution to convert RefineMask pretrained .pth model to .onnx model or other formats?
Or, if anyone knows, could you tell me the implementation to other train/inference platforms?

[Gang Zhang]

ut the result is failed.
I also tried with MMdeploy v0.14.0 but the result is fa

Unfortunately, we did not test the deployment process for ONNX.

I can transfer the code to the latest MMDetection if I have time later.

[RiverLight4]

@zhanggang001 , Thanks for your reply!

Unfortunately, we did not test the deployment process for ONNX.
I can transfer the code to the latest MMDetection if I have time later.

OK, I'll wait for your implement. I'll try to challenge converting by myself, too.
I hope that RefineMask can be used in other inference engines.

Additional info:

It seems that RefineMask is able to work on MMDetection v2.28.2 with little bit fix below.
However, I'm facing the trouble that torch.jit.trace cannot relay 'img_metas' to simple_test_mask in mmdet/models/roi_heads/refine_roi_head.py.

ori_shape = img_metas[0]['ori_shape'] ## ERROR: 'img_metas'[0] has no 'ori_shape', 'scale_factor', etc. because 'img_metas' cannot input into torch.jit.trace()

I'll ask about this problem at open-mmlab/mmdeploy community, because I think it's not the problem of RefineMask but would be related to MMdeploy or pytorch 1.13.x.

My workaround fix info with MMdetection 2.28.2

1. fix refine_roi_head.py : def simple_test_mask : (workaround)

• det_bboxes -> det_bboxes [0]
• det_labels -> det_labels [0]
  • Because det_bboxes and det_labels are provided as list in MMDetection-2.28.2, for inference multiple images.
  • This fix causes that, if multiple images inputs, only 1st image is inferenced. not good patch :(

2. fix configs/refinemask/r50-refinemask-1x.py

• Move train_cfg and test_cfg into MaskRCNN model layer. (same level as backbone, neck, etc.)
  • see here
• train_cfg/rpn_proposal, test_cfg/rpn : nms_thr = 0.7 -> nms=dict(type='nms', iou_threshold=0.7)
  • see open-mmlab/mmdetection/pull/4636

[Myxrf]

Hello, have you successfully exported onnx？

[RiverLight4]

Unfortunately, No.