Cuda out of memory
yyash-patel opened this issue · comments
Hi !
I am using 4 Quadro RTX 6000 (24 GB each) and Cuda goes out of memory, with the default Monoscene model provided to train on nuscenes dataset with image size 1600x900.
Could you tell me about your memory consumption of GPU when training Nuscenes with the Monoscene model with the original image resolution - 1600x900 to get a better understanding of the problem?
Thanks,
Yash
From our experience, each batch may take around 40GB of GPU memory. In your case, it's recommended to downsample the image to fit in the 24GB memory.