Comparison with IPT
yifanjiang19 opened this issue · comments
Hi,
Thanks for sharing this interesting work. Table. 6, CBSD68, sigma=50 shows that IPT achieves 28.39 PSNR. However, the original paper of IPT shows that it can achieves 29.88 (in their Table. 2). Is there any difference with these two settings?
Yes, there are some difference. We notice that IPT reported Y-channel PSNR for color image denoising. Since papers in the literature often report RGB PSNR, we re-test IPT and report its RGB PSNR as well.
But their competitors reported in their Table 2 is RGB PSNR (such as DnCNN and RDN). Shouldn't that be an unfair comparion?
https://github.com/cszn/DPIR#denoising-results-on-bsd68-and-urban100-datasets
Denoising results on BSD68 and Urban100 datasets
| Dataset | Noise Level | FFDNet-PSNR(RGB) | FFDNet-PSNR(Y) | DRUNet-PSNR(RGB) | DRUNet-PSNR(Y) |
|---|---|---|---|---|---|
| CBSD68 | 30 | 30.32 | 32.05 | 30.81 | 32.44 |
| CBSD68 | 50 | 27.97 | 29.65 | 28.51 | 30.09 |
| Urban100 | 30 | 30.53 | 32.72 | 31.83 | 33.93 |
| Urban100 | 50 | 28.05 | 30.09 | 29.61 | 31.57 |
@cszn Thanks for your response! I think your table provide a reasonable number. My only concern is whether IPT report RGB results or Y results. In there Table. 2 (attached below), FFDNet achieves 27.96 db on CBSD68-sigma50, which is consistent with RGB results on your table. Based on these results, I think their table. 2 (attached) reports RGB results, unless their results are incorrect.
Any new updates? @JingyunLiang @cszn , I believe the comparison with IPT could be important in this work. And many followers will use the reported number in their work.
You can run the testing codes of both methods.
I think the answer is very clear. In Table 2 of IPT, the compared methods are measured by color PSNR, while IPT is measured by Y-channel PSNR (always much higher than color PSNR). I recommend following papers cite the color PSNR (the correct one) reported in our paper.
Yes, there are some difference. We notice that IPT reported Y-channel PSNR for color image denoising. Since papers in the literature often report RGB PSNR, we re-test IPT and report its RGB PSNR as well.
Thanks for further clarifying. I will contact the author of IPT for further details. If that is reported on Y-channel, I think it will confuse the followers.
After I contact the original author of IPT. I think they misunderstand the results from RDN and DnCNN and give an unfair evaluation on their Table. 2. Thanks again for clear clarification and patient discussion here. Hope to see more interesting work from you in the future!