Don't know why it wont work (killed)
krawek opened this issue · comments
Everything is setup properly to the best of my knowledge, when I start the training I simply get a Killed message and it stops.
Is there a way to see a log or is this a known issue?
running on a 3090.
I'm noticing Memory maxing out and then the Killed happens, is there a memory requirement?
I've played with the batch size and such in the .gin file to see if it alleviates it but with no luck.
Currently i have 32GB of RAM and 24GB VRAM
Solved it by adding more memory, so hardware limitation on my end, closing.
So, is it possible to load data and do sampling directly on GPU? Because it's really consumes a lot of CPU memory but very little on the GPU.
Solved it by adding more memory, so hardware limitation on my end, closing.
is it possible to run it on 30GB RAM and 6 GB VRAM?
any suggestion to train on low end gpu?