The experimental results could not be reproduced
JJJTF opened this issue · comments
I wonder why the CIFAR100 result I reproduced was only 57%, but the experimental result of the paper was 75%. I haven't found out why.
Hi @JJJTF, thanks for your interest! Could you please be more specific on from which model that you cannot reproduce the results? And also your experiment settings, e.g. number of GPUs, hyperparameters like learning rate and batch size. If you follow the default hyperparameter settings in this repo, you should get the similar results.
my experiment settings is hvt-s-1.json file and cuda:0,1, others is default.
Hi @JJJTF, I just trained this model on 2 V100 GPUs. My python environment has PyTorch 1.7.1 and CUDA 10.1. Without modifying any code in this repo, I got the final accuracy of 73.68% by using the following commands for training,
python -m torch.distributed.launch --nproc_per_node=2 --use_env main.py --config config/hvt-s-1.json --data-set CIFAR --data-path [path/to/cifar100]Note that your final result could be slightly different based on your environment, but ideally it should not be significantly lower, i.e. 57% is not normal. It would be better if you can double-check your code.
Here is the training log for your reference: all_logs.txt
Thank you very much for your timely answer to each question. I will run again according to your all_logs.txt parameter.