Cuda out of memory when running tools/test.py

[Fatma Mazen]

Dear sir,
Thank you very much for sharing your work
I have trained a model using a lightweight version of solov2
When trying to test it by running tools/test.py, I got CUDA out of memory only after processing half images of my test set as shown in the attached message
Could you kindly help me to solve this issue?
loading annotations into memory...
Done (t=0.16s)
creating index...
index created!
[ ] 0/8512, elapsed: 0s, ETA:/opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3613: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode)
/opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3658: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details.
"The default behavior for interpolate/upsample with float scale_factor changed "
[>>>>>>>>>>>>>>>>> ] 4833/8512, 28.7 task/s, elapsed: 168s, ETA: 128sTraceback (most recent call last):
File "/kaggle/working/SOLO/tools/test.py", line 282, in
main()
File "/kaggle/working/SOLO/tools/test.py", line 241, in main
outputs = single_gpu_test(model, data_loader, args.show)
File "/kaggle/working/SOLO/tools/test.py", line 26, in single_gpu_test
result = model(return_loss=False, rescale=not show, **data)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
return self.module(*inputs[0], **kwargs[0])
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/kaggle/working/SOLO/mmdet/core/fp16/decorators.py", line 49, in new_func
return old_func(*args, **kwargs)
File "/kaggle/working/SOLO/mmdet/models/detectors/base.py", line 144, in forward
return self.forward_test(img, img_meta, **kwargs)
File "/kaggle/working/SOLO/mmdet/models/detectors/base.py", line 127, in forward_test
return self.simple_test(imgs[0], img_metas[0], **kwargs)
File "/kaggle/working/SOLO/mmdet/models/detectors/single_stage_ins.py", line 92, in simple_test
seg_result = self.bbox_head.get_seg(*seg_inputs)
File "/kaggle/working/SOLO/mmdet/models/anchor_heads/solov2_head.py", line 381, in get_seg
featmap_size, img_shape, ori_shape, scale_factor, cfg, rescale)
File "/kaggle/working/SOLO/mmdet/models/anchor_heads/solov2_head.py", line 482, in get_seg_single
seg_masks = seg_masks > cfg.mask_thr
RuntimeError: CUDA out of memory. Tried to allocate 40.00 MiB (GPU 0; 15.90 GiB total capacity; 14.73 GiB already allocated; 31.75 MiB free; 14.97 GiB reserved in total by PyTorch)

[Fatma Mazen]

I have solved the problem by using test_ins.py instead of test.py since I have only one gpu