LovaszSoftmax Loss test case not working
omerferhatt opened this issue · comments
When I try to run your test case to check it out. It throws error.
terminate called after throwing an instance of 'thrust::system::system_error'
what(): CUDA free failed: cudaErrorIllegalAddress: an illegal memory access was encountered
I'm using Python 3.7.10 and PyTorch 1.8.1. GPU is Nvidia RTX 3070.
Hi,
I do not have experience with 3070 or cuda 11. Can your pytorch work well with pytorch native ops ? This problem can also be caused by numeric stablillity, would you please try with smaller lr or less regularization ?
Yes, pytorch is working fine with the all other stuffs. It seems LovaszSoftmaxV1 working okay with both forward and backward but when I try to execute forward pass on V3 it gives the same error. And I changed lr and loss reg. too.
It is a conflict problem of pytorch, you can install nighly-version of pytorch and see if there is still this problem. see this: pytorch/pytorch#52663
It's working with pytorch 1.9.0 cu11.1