A MXNet implementation of Squeeze-and-Excitation Networks (SE-ResNext 18, 50, 101, 152)
This is a MXNet implementation of Squeeze-and-Excitation Networks (SE-ResNext and SE-Resnet) architecture as described in the paper Squeeze-and-Excitation Networks proposed by Jie Hu et. al. They deployed this SE block in SENet and win the Imagenet 2017 classification task.
The author's caffe implementation can be found in his repo on GitHub.
This is an illustration of a Squeeze-and-Excitation block.
The SE-ResNet module is implemented as followed:
The SE-ResNext 50 is implemented following this table:
This MXNet implementation is refered to taki0112's tensorflow version. I also refered a PyTorch implementation from kuangliu. BTW, I add a dropout layer before the last FullyConnected layer. I attach the training code if you want to train your own data with SE-ResNext architecture by yourself.
Install MXNet(0.11.0) on GPUs mechine with NVIDIA CUDA 8.0, and it's better also installed with cuDNN v6 or later version (I'm not testing cuDNN v7).
ImageNet'12 dataset
Imagenet 1000 class dataset with 1.2 million images. Because this dataset is about 120GB, so you have to download by yourself. Sorry for this inconvenience.
For data preparation, you can refer my pervious part of densenet or you can also visit the repo of Wei Wu. In his page, there is a very detailed information about how to prepare your data.
When you finised data preparation, please make sure the data locates the same folder of source codes. You also need to change path of path_imgrec in line 84 and line 108 of train_xception.py. Then you can run the training cmd just like this (here, I use 4 gpus for training):
python -u train_se_resnext_w_d.py --data-dir data/imagenet --data-type imagenet --depth 50 --batch-size 192 --num-group 64 --drop-out 0.0 --gpus=6,7,8,9
Maybe you should change batch-size from 256 to 128 due to the memory size of GPU.
When we want to train the large dataset and hope to change learning rate manually, or the machine is suddenly shutdown due to some reason, of course, we definitely hope we can continue to train model with previous trained weights. Then, your can use this cmd:
python -u train_se_renext_w_d.py --data-dir data/imagenet --data-type imagenet --depth 50 --batch-size 192 --num-group 64 --gpus=0,1,2,3 --model-load-epoch=50 --lr 0.001 --retrain
This means you can retrain your xception model from epoch 50 and change lr=0.001 using 4 GPUs.
The training procedure is ongoing. So, I hope anyone who are mxnet fun can test this code with me. When I finish, I will update more information about training and validation.
I update the learning curves of SE-Resnext 50 (batchsize=192) trained on vggface datasets with the comparison of Resnext 50 (batchsize=256) (Updated at Oct-7, 2017).
Added SE-Resnet 18, 50, 101, 152 (Updated at Sep-27, 2017).
Gluon version is coming soon.
[1] Jie Hu, Li Shen and Gang Sun. "Squeeze-and-Excitation Networks"
[2] Tensorflow implementation of SENet from taki0112's