Onnx exported model only provides empty output
ashnair1 opened this issue · comments
Using this fork, I was able to successfully export my Faster RCNN (ResNext-101_64x4d) to onnx. However, when I try to do inference, it always yields empty outputs.
Here are my conversion logs:
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/models/detectors/base.py:147: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
imgs_per_gpu = int(imgs[0].size(0))
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:55: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
start_tensor = torch.as_tensor(start, dtype=torch.long, device=device)
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:56: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
end_tensor = torch.as_tensor(end, dtype=torch.long, device=device)
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/models/anchor_heads/rpn_head.py:67: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert rpn_cls_score.size()[-2:] == rpn_bbox_pred.size()[-2:]
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:95: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
k = torch.tensor([k], dtype=torch.long)
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:105: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
values, keep = torch.topk(x, n, dim=dim, **kwargs)
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/bbox/transforms.py:51: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
min_val = torch.as_tensor(min, dtype=dtype, device=device)
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/bbox/transforms.py:56: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
max_val = torch.as_tensor(max, dtype=dtype, device=device)
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:64: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
for i in range(num_classes):
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:66: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if not cls_inds.any():
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:69: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if multi_bboxes.shape[1] == 4:
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/ops/nms/nms_wrapper.py:50: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if dets_th.shape[0] == 0:
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if bboxes.shape[0] > max_num:
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/bbox/transforms.py:193: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if bboxes.size(0) > 0:
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/models/roi_extractors/single_level.py:72: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
eps = torch.tensor([1e-6], dtype=scale.dtype, device=scale.device)
ONNX model has been saved to "/home/ashwin/Desktop/Projects/mmdetection/demo/onnx_test/faster_rcnn_x101_64x4d_fpn_1x.onnx"
ONNX check passed.
And this is my inference method:
import numpy as np
import cv2
import onnx
import onnxruntime
ort_session = onnxruntime.InferenceSession("faster_rcnn_x101_64x4d_fpn_1x.onnx")
# test a single image array
img = 'test.png'
img = cv2.imread(img)
img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
#img = np.random.rand(1024,1024,3)
# HWC -> CHW
img = np.transpose(img, (2, 0, 1)).astype(np.float32)
# CHW -> NCHW
img = np.array([img])
# compute ONNX Runtime output prediction
ort_inputs = {ort_session.get_inputs()[0].name: img}
ort_outs = ort_session.run(None, ort_inputs)
print("DONE")
>> ort_outs
[array([], shape=(0, 5), dtype=float32), array([], dtype=int64)]
What I see, is that your code is lacking data pre-processing. This Faster R-CNN model expects to get properly resized and normalized RGB image as an input. All pre-processing parameters can be found in config file under the test_pipeline section.
As a quick model correctness check, if you have COCO dataset on your machine, try running tools/test_exported.py script for the ONNX model with a --show key to get a visualization of detection results.
This model wasn't trained on COCO but a proprietary dataset.
I actually tried a stripped down version of your test_exported.py to check but the results were the same.
from mmdet.utils.deployment import ModelONNXRuntime
import mmcv
import cv2
import numpy as np
cfg = "./faster_rcnn_x101_64x4d_fpn_1x.py"
model = "./faster_rcnn_x101_64x4d_fpn_1x.onnx"
cfg = mmcv.Config.fromfile(cfg)
cfg.model.pretrained = None
cfg.data.test.test_mode = True
classes = ('A', 'B', 'C', 'D', 'E', 'F')
model = ModelONNXRuntime(model, cfg=cfg, classes=classes)
# test a single image array
img = './test.png'
img = cv2.imread(img)
img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
#img = np.random.rand(1024,1024,3)
img = cv2.resize(img, (1024, 1024))
# HWC -> CHW
img = np.transpose(img, (2, 0, 1)).astype(np.float32)
# CHW -> NCHW
img = np.array([img])
result = model(img)
>>> result
{'boxes': array([], shape=(0, 5), dtype=float32),
'labels': array([], dtype=int64)}
With my current config and model checkpoint (.pth file), inference worked inside mmdetection i.e. using inference_demo.ipynb
Edit: Oh you mean use the test pipeline within mmdetection.
Correct.
It should be something like this:
test_pipeline = [LoadImage()] + cfg.data.test.pipeline[1:]
test_pipeline = Compose(test_pipeline)
# test a single image array
img = './test.png'
img = cv2.imread(img)
img = test_pipeline(dict(img=img))['img'][0].numpy()
img = img[None, :, :, :]
result = model(img)This way pre-processing part still depends on PyTorch, but usually it's pretty straightforward to re-implement it for a particular model at hand in a pure numpy or whatever.
And, by the way, I believe that tools/test_exported.py script should work even for a custom dataset .
Brilliant! That worked perfectly. Thanks a ton!
Using this fork, I was able to successfully export my Faster RCNN (ResNext-101_64x4d) to onnx. However, when I try to do inference, it always yields empty outputs.
Here are my conversion logs:
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/models/detectors/base.py:147: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! imgs_per_gpu = int(imgs[0].size(0)) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:55: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. start_tensor = torch.as_tensor(start, dtype=torch.long, device=device) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:56: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. end_tensor = torch.as_tensor(end, dtype=torch.long, device=device) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/models/anchor_heads/rpn_head.py:67: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert rpn_cls_score.size()[-2:] == rpn_bbox_pred.size()[-2:] /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:95: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. k = torch.tensor([k], dtype=torch.long) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:105: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! values, keep = torch.topk(x, n, dim=dim, **kwargs) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/bbox/transforms.py:51: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. min_val = torch.as_tensor(min, dtype=dtype, device=device) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/bbox/transforms.py:56: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. max_val = torch.as_tensor(max, dtype=dtype, device=device) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:64: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! for i in range(num_classes): /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:66: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if not cls_inds.any(): /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:69: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if multi_bboxes.shape[1] == 4: /home/ashwin/Desktop/Projects/mmdetection2/mmdet/ops/nms/nms_wrapper.py:50: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if dets_th.shape[0] == 0: /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if bboxes.shape[0] > max_num: /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/bbox/transforms.py:193: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if bboxes.size(0) > 0: /home/ashwin/Desktop/Projects/mmdetection2/mmdet/models/roi_extractors/single_level.py:72: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. eps = torch.tensor([1e-6], dtype=scale.dtype, device=scale.device) ONNX model has been saved to "/home/ashwin/Desktop/Projects/mmdetection/demo/onnx_test/faster_rcnn_x101_64x4d_fpn_1x.onnx" ONNX check passed.And this is my inference method:
import numpy as np import cv2 import onnx import onnxruntime ort_session = onnxruntime.InferenceSession("faster_rcnn_x101_64x4d_fpn_1x.onnx") # test a single image array img = 'test.png' img = cv2.imread(img) img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB) #img = np.random.rand(1024,1024,3) # HWC -> CHW img = np.transpose(img, (2, 0, 1)).astype(np.float32) # CHW -> NCHW img = np.array([img]) # compute ONNX Runtime output prediction ort_inputs = {ort_session.get_inputs()[0].name: img} ort_outs = ort_session.run(None, ort_inputs) print("DONE")>> ort_outs [array([], shape=(0, 5), dtype=float32), array([], dtype=int64)]
Using this fork, I was able to successfully export my Faster RCNN (ResNext-101_64x4d) to onnx. However, when I try to do inference, it always yields empty outputs.
Here are my conversion logs:
/home/ashwin/Desktop/Projects/mmdetection2/mmdet/models/detectors/base.py:147: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! imgs_per_gpu = int(imgs[0].size(0)) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:55: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. start_tensor = torch.as_tensor(start, dtype=torch.long, device=device) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:56: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. end_tensor = torch.as_tensor(end, dtype=torch.long, device=device) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/models/anchor_heads/rpn_head.py:67: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert rpn_cls_score.size()[-2:] == rpn_bbox_pred.size()[-2:] /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:95: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. k = torch.tensor([k], dtype=torch.long) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/utils/misc.py:105: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! values, keep = torch.topk(x, n, dim=dim, **kwargs) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/bbox/transforms.py:51: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. min_val = torch.as_tensor(min, dtype=dtype, device=device) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/bbox/transforms.py:56: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. max_val = torch.as_tensor(max, dtype=dtype, device=device) /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:64: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! for i in range(num_classes): /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:66: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if not cls_inds.any(): /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:69: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if multi_bboxes.shape[1] == 4: /home/ashwin/Desktop/Projects/mmdetection2/mmdet/ops/nms/nms_wrapper.py:50: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if dets_th.shape[0] == 0: /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/post_processing/bbox_nms.py:85: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if bboxes.shape[0] > max_num: /home/ashwin/Desktop/Projects/mmdetection2/mmdet/core/bbox/transforms.py:193: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if bboxes.size(0) > 0: /home/ashwin/Desktop/Projects/mmdetection2/mmdet/models/roi_extractors/single_level.py:72: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. eps = torch.tensor([1e-6], dtype=scale.dtype, device=scale.device) ONNX model has been saved to "/home/ashwin/Desktop/Projects/mmdetection/demo/onnx_test/faster_rcnn_x101_64x4d_fpn_1x.onnx" ONNX check passed.And this is my inference method:
import numpy as np import cv2 import onnx import onnxruntime ort_session = onnxruntime.InferenceSession("faster_rcnn_x101_64x4d_fpn_1x.onnx") # test a single image array img = 'test.png' img = cv2.imread(img) img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB) #img = np.random.rand(1024,1024,3) # HWC -> CHW img = np.transpose(img, (2, 0, 1)).astype(np.float32) # CHW -> NCHW img = np.array([img]) # compute ONNX Runtime output prediction ort_inputs = {ort_session.get_inputs()[0].name: img} ort_outs = ort_session.run(None, ort_inputs) print("DONE")>> ort_outs [array([], shape=(0, 5), dtype=float32), array([], dtype=int64)]
Did you follow the instal.md to install this fork ? it is nessecery to install openvino?
- I can confirm that install instructions are up-to-date.
- OpenVINO is optional, but recommended dependency.
I use pip install -i https://pypi.tuna.tsinghua.edu.cn/simple/ -r requirements.txt --timeout=10000 but got ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall. It seem requirements.txt get conflict about PyYAML. could you give me some advice ?
Are you using a virtual environment?
Are you using a virtual environment?
yeah.I use it in docker...... I just remove PyYAML from requirements/option.txt . Now it seems works. I will try.
Cool.
BTW, the Dockerfile is also up-to-date, if you need it.
Thank you for your prompt reply. Now I trying to use it to training Retinanet and export its onnx model. Hope I can get a good result. Thanks a lot.
Cool.
BTW, the Dockerfile is also up-to-date, if you need it.
Your work is so great. I got the result from the onnx model is much similiar to the mmdet output.
Cool.
BTW, the Dockerfile is also up-to-date, if you need it.
It seem pytorch ==1.4.0 is not necessary. I test your work on pytorch 1.3.0 and it works.In addition. maybe your could motify the export onnx code and add ONNX's Optimization ？ I'm trying to use ONNX's Optimization to avoid some onnxruntimewarning by mmdet/tools/pytorch2onnx.py but it didn't work.
File "/home/toor/QT_Python/CVLIDA/testtrain.py", line 280, in export_onnx_model
pass_model(onnx_model_path)
File "/home/toor/QT_Python/CVLIDA/testtrain.py", line 259, in pass_model
onnx_model = optimizer.optimize(onnx_model, passes)
File "/home/toor/anaconda3/envs/pyside/lib/python3.6/site-packages/onnx/optimizer.py", line 55, in optimize
optimized_model_str = C.optimize(model_str, passes)
IndexError: _Map_base::at
could you give me some advice?
this is the pass_model func
def pass_model(export_name):
onnx_model = onnx.load(export_name)
passes = ['eliminate_unused_initializer']
if passes is not None:
all_passes = optimizer.get_available_passes()
assert all(p in all_passes for p in passes),
f'Only {all_passes} are supported'
onnx_model = optimizer.optimize(onnx_model, passes)
onnx.save(model, export_name)
Cool.
BTW, the Dockerfile is also up-to-date, if you need it.
I fix it through motify the model.export(
**data,
export_name=export_name,
verbose=verbose,
opset_version=opset,
strip_doc_string=strip_doc_string,
operator_export_type=torch.onnx.OperatorExportTypes.ONNX,
input_names=['image'],
output_names=output_names,
dynamic_axes=dynamic_axes,
keep_initializers_as_inputs=True). add a param keep_initializers_as_inputs=True can fix this. hope you could update it.