Giters

yitu-opensource / MobileNeXt

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Reproducing the results of using scripts

jbr97 opened this issue · comments

commented

Hey there,

Unfortunately, I've found something confused when reproducing the results of using scripts mnext_mbv2.sh: MobileNeXt large model based on MobileNetV2 backbone.
I changed the ImageNet path written in the scripts:

~/WSNet_improvement/imagenet/ILSVRC/Data/CLS-LOC/ -> /my/path/to/imagenet/ILSVRC2012/

Then I ran the following commands as mentioned in the README.md (p.s. in a pytorch1.5.1 environment with 4 NVIDIA V100s):

bash mnext_mbv2.sh 4

And I found the codes loaded the checkpoint from 'output/74_09_i2rnetv3_half_id/model_best.pth.tar (epoch 197)' but got '*** Best metric: 69.21200000732422 (epoch 209)', I think this result is not corresponding to any results in papers.

Did I do something wrong or misunderstand something when executing the script used for the training from scratch?
Could you please show more details about the scripts used for the training from scratch?

Thanks in advance!

commented

Hey there,

Unfortunately, I've found something confused when reproducing the results of using scripts mnext_mbv2.sh: MobileNeXt large model based on MobileNetV2 backbone.
I changed the ImageNet path written in the scripts:

~/WSNet_improvement/imagenet/ILSVRC/Data/CLS-LOC/ -> /my/path/to/imagenet/ILSVRC2012/

Then I ran the following commands as mentioned in the README.md (p.s. in a pytorch1.5.1 environment with 4 NVIDIA V100s):

bash mnext_mbv2.sh 4

And I found the codes loaded the checkpoint from 'output/74_09_i2rnetv3_half_id/model_best.pth.tar (epoch 197)' but got '*** Best metric: 69.21200000732422 (epoch 209)', I think this result is not corresponding to any results in papers.

Did I do something wrong or misunderstand something when executing the script used for the training from scratch?
Could you please show more details about the scripts used for the training from scratch?

Thanks in advance!

Hi, the mismatch comes from the mismatch between the model definition file and the model weights. The script you are using is for full identity tensor loading. To load the half identity tensor weights, you need to modify the model definition file line 73 and change the div to be 2. Then you should get a proper accuracy. Please refer to the link below:

https://github.com/zhoudaquan/rethinking_bottleneck_design/blob/c61ebec6f007257274ac25590a17063abc1a4645/mobilenext/codebase/models/mnext.py#L73

Sorry that I did not pull out this as an argument yet. Will fix it later.

commented

Solved, many thanks