pretrained SVHN
liangbright opened this issue · comments
Thank you for providing the models.
Could you please check the pretrained SVHN?
The accuracy of the pretrained SVHN is close to zero.
Can confirm, 0.47% on my machine.
do you guys use single digit？ it literally doesn't work.
real target [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
prediction [9, 0, 1, 2, 3, 4, 5, 6, 7, 8]
Re-adjust the prediction results. In my case, the model can achieve 98% training accuracy and 96% test accuracy.
Note that the range of SVHN label is [-1, 8], but the predicted value is from 0 to 9. Only fix it in line 190 of misc.py (only for svhn dataset):
idx_pred = idx_pred
to
idx_pred = idx_pred - 1 # modified
The top1 accuracy could achieve more than 90% using 8-bit quantization.
The problem comes from the target_transform function under svhn/dataset.py. Replace the implementation with the following snippet and you should be fine.
return (int(target) - 1) % 10