WXinlong / SOLO

SOLO and SOLOv2 for instance segmentation, ECCV 2020 & NeurIPS 2020.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

RuntimeError: There were no tensor arguments to this function

DongGuangchang opened this issue · comments

Traceback (most recent call last):
File "tools/train.py", line 125, in
main()
File "tools/train.py", line 121, in main
timestamp=timestamp)
File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/apis/train.py", line 111, in train_detector
timestamp=timestamp)
File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/apis/train.py", line 297, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/runner.py", line 364, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/runner/runner.py", line 268, in train
self.model, data_batch, train_mode=True, **kwargs)
File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/apis/train.py", line 78, in batch_processor
losses = model(**data)
File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/dy/software/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/core/fp16/decorators.py", line 49, in new_func
return old_func(*args, **kwargs)
File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/models/detectors/base.py", line 142, in forward
return self.forward_train(img, img_meta, **kwargs)
File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/models/detectors/single_stage_ins.py", line 78, in forward_train
*loss_inputs, gt_bboxes_ignore=gt_bboxes_ignore)
File "/home/dy/dong/instanceSegmentation/SOLO-master/mmdet/models/anchor_heads/solov2_head.py", line 249, in loss
loss_ins = torch.cat(loss_ins).mean()
RuntimeError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::_cat. This usually means that this function requires a non-empty list of Tensors. Available functions are [CUDATensorId, CPUTensorId, VariableTensorId]

I encountered a problem during training. How can I solve it? thanks!

I hit the same issue. The problem is that there were empty predictions from the model for that batch which returns an empty list of losses. Not sure how this could be solved.
This is a temporary solution that you could try (added to SOLO/mmdet/models/anchor_heads/solov2_head.py:249):

        if len(loss_ins) == 0:
            loss_ins = torch.tensor(0.0, device=gt_bbox_list[0].device)
        else:
            loss_ins = torch.cat(loss_ins).mean()
            loss_ins = loss_ins * self.ins_loss_weight