Worse results by reproducing MoCo, InsDIS and CMC on ImageNet100

[nkliuyifang]

Hi @HobbitLong,

Thanks for your nice paper and publlic code!

I have reproduced results of MoCo and InsDIS on ImageNet100 following your steps.
I got 67.44 for MoCo and 66.02 for InsDIS, which are worse than the expected 73.4 and 69.1.

Could you please help me about this?

Best
Mengyuan

[Yonglong Tian]

Hi, @nkliuyifang ,

Can you share more details? e.g., training command line, testing command line.

How is the result for CMC on your end? I want to see if this is a common issue for all of the three or only InsDis and MoCo.

[nkliuyifang]

Hi, @HobbitLong ,

The best result of CMC using "--view YCbCr" and "--nce_k 16384" is 70.54

The corresponding training command line is:

The corresponding testing command line is:

All related results of InsDIS, MoCo and CMC on ImageNet100 dataset is listed as follows, where the training set of ImageNet100 is used for both unsupervised learning and fine-tuning, and the testing set of ImageNet100 is used for testing classifiaction results.

[Yonglong Tian]

Hi, @nkliuyifang,

I can see several gaps as below, which apply to all MoCo, InsDis, and CMC:
(1) For pre-training on ImageNet100, use batch_size 128 instead of 256. This is equivalent to increase the learning rate a bit. For full ImageNet, keep using 256.
(2) Switch from NCE loss to softmax-ce for pre-training
(3) Use a learning rate of 10 for linear probing stage, as can be found in README

[nkliuyifang]

Hi, @HobbitLong,

Thanks! I will modify these settings and try again.

[nkliuyifang]

I have reached the expected result. Thanks!