training not proceed after checkpoint saving

Question

training not proceed after checkpoint saving

jhkwag970 opened this issue 4 months ago · comments

Hello, Thank you for nice work and sharing with us!

As I am working through training I encountered problem.

Training does not proceed after this point. I check the logs of your pre defined model and after saving checkpoint it should not taking too long to start training.

My current version of MMCV and python are

Python: 3.8.18
PyTorch: 1.9.1+cu111
TorchVision: 0.10.1+cu111
OpenCV: 4.6.0
MMCV: 1.4.0
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.1
MMDetection: 2.19.0
MMSegmentation: 0.19.0
MMDetection3D: 0.17.2+e03f097

and using 3 RTX 3090.

Any help would be appreciated!!