Regarding multi GPU training
Yifehuang97 opened this issue · comments
I apologize for reaching out again, but I've encountered an error in the diff-gaussian-rasterization while using multiple GPUs(Vanilla Torch DDP). It appears that the rasterizer's device may not be correctly configured, resulting in an illegal memory access error on all ranks except rank 0.
Have you experienced this issue before? If so, could you share how you managed to resolve it?
Thank you for your time and assistance!
Sorry, I have not trained on multiple GPUs. Maybe you can find some solutions on other repos for multi-GPU training with diff-gaussian-rasterization.
Thanks for your reply! I've solved it by using torch lightning.