Image dimentions - out of memory error while training
opened this issue · comments
Hai,
I am trying to train day to night translation with my own data-set. The image dimension in both domains are 1280x720, but in the given config file the image is cropped to 256x256, after training with 256x256 random cropping the output images generated are of dimension 420x256. I need to train this model with 512x512 and I have changed in the data_loader also since i am having a cuda:out_of_memory issue after 100 iteration. why this happens like this..??why we need to do training only with 256x256 random cropping only..??This makes my output video to blur and breaking of pixels. I need to get rid of that.
How should I train with the whole image dimension and the generated output images are same as that of the input image..??
If you have a GPU with 16GB, you could probably train to generate 512x512 images. Note that the batch size has to be set to 1. NVIDIA has some GPU cards that have 48GB+ GPU memory. You might need those cards to generate HD+ images.