laisimiao / classification-cifar10-pytorch

Train several classical classification networks in cifar10 dataset by PyTorch

Classification-cifar10-pytorch

I am testing several classical classification networks performance on cifar10 dataset by PyTorch! Chinese blog

Requirements

• pytorch
• torchsummary
• python3.x

Training

• Single-GPU

  # Start training with: 
  python main.py
  
  # You can manually resume the training with: 
  python main.py --resume --lr=0.01

• Muiti-GPU

  # Start training with(assume you have 2 GPUs): 
  python -m torch.distributed.launch --nproc_per_node=2 main.py
  # You can manually resume the training with:
  python -m torch.distributed.launch --nproc_per_node=2 main.py --resume

Results

Model	My Acc.	Total params	Estimated Total Size (MB)	Trainable params	Params size (MB)	Saved model size (MB)	GPU memory usage(MB)
MobileNetV2	92.64%	2,296,922	36.14	2,296,922	8.76	8.96	3107
VGG16	94.27%	14,728,266	62.77	14,728,266	56.18	59.0	1229
PreActResNet18	94.70%	11,171,146	53.38	11,171,146	42.61	44.7	1665
ResNeXt29(2x64d)	95.09%	9,128,778	99.84	9,128,778	34.82	36.7	5779
ResNet50	95.22%	23,520,842	155.86	23,520,842	89.72	94.4	5723
DPN92	95.42%	34,236,634	243.50	34,236,634	130.60	137.5	10535
ResNeXt29(32x4d)	95.49%	4,774,218	83.22	4,774,218	18.21	19.2	5817
DenseNet121	95.55%	6,956,298	105.05	6,956,298	26.54	28.3	8203
ResNet18	95.59%	11,173,962	53.89	11,173,962	42.63	44.8	1615
ResNet101	95.62%	42,512,970	262.31	42,512,970	162.17	170.6	8857

Note:

1. Above GPU memory usage(MB) was observed with batch size=128.
2. For PreActResNet18, I set initial learning rate=0.1, but it can't converge, so I set it's initial lr=0.01.
3. I firstly train VGG16, ResNet18 and ResNet50 with total epochs=400. But I want to get results earlier, so for remaining networks, I set total epochs=300 (besides, afterwards it just improve a little).
4. Run the scripts: python main.py --resume --lr 0.1 --trainbs 128 --testbs 100

Pre-trained models

You can obtain pre-traind models(as above list) from here: [Baidu Drive] [Google Drive]