What about the performance of this reproduction?
Wuziyi616 opened this issue · comments
Hi! Thank you for your great work! Recently I'm also trying to reproduce PU-Net in pytorch and came across your repo. I just wonder what performance did you get using this repo? Is it competitive with the original paper? Thanks!
In this repo, results of the baseline model are (NUC) 0.2396 0.1992 0.1764 0.1620 0.1536 0.1482 with different p (0.2% 0.4% 0.6% 0.8% 1.0% 1.2%, respectively). The official results are 0.174 0.138 0.122 0.115 0.112 0.110. For NUC, the lower is better. Note that we evaluated the NUC only on 40 disks (9000 in the paper, section 4.3) since it took us a very long time. For full evaluation, please refer to this issue yulequan/PU-Net#14.
Furthermore, we also provide the EMD and CD evaluated on test split (on patches, not on complete objects).
EMD: 0.110 * 10^2
CD: 0.305 * 10^3
I will update the README later.
Thank you so much for your quick reply! BTW, would you mind uploading the weight for the model you trained which gets the result you mentioned? It will be helpful because I won't need to download the training data and train it by myself (I just want to use a pre-trained PU-Net for other applications). Thanks!
Model weight for the baseline model is uploaded to Google Drive. Please feel free to contact me if you encounter any problem.
Got it! Thank you so much for your help!