Octave Convolution
MXNet implementation for:
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
Note:
- This repo is under development.
To Do List
- Code for ablation study (by Symbol API)
- Code for the rest exps (by Gluon API)
- Training script
- Training logs
- Trained models
ImageNet
Ablation
- Loss: Softmax
- Learning rate: Cosine (warm-up: 5 epochs, lr: 0.4)
- MXNet API: Symbol API
Note:
- All residual networks in ablation study adopt pre-actice version[1] for convenience.
Others
- Learning rate: Cosine (warm-up: 5 epochs, lr: 0.4)
- MXNet API: Gluon API
| Model | alpha | label smoothing[2] | mixup[3] | #Params | #FLOPs | Top1 |
|---|---|---|---|---|---|---|
| 0.75 MobileNet (v1) | .375 | 2.6 M | 213 M | 70.6 | ||
| 1.0 MobileNet (v1) | .5 | 4.2 M | 321 M | 72.4 | ||
| 1.0 MobileNet (v2) | .375 | Yes | 3.5 M | 256 M | 72.0 | |
| 1.125 MobileNet (v2) | .5 | Yes | 4.2 M | 295 M | 73.0 | |
| Oct-ResNet-152 | .125 | Yes | Yes | 60.2 M | 10.9 G | 81.4 |
| Oct-ResNet-152 + SE | .125 | Yes | Yes | 66.8 M | 10.9 G | 81.6 |
Citation
@article{chen2019drop,
title={Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution},
author={Chen, Yunpeng and Fan, Haoqi and Xu, Bing and Yan, Zhicheng and Kalantidis, Yannis and Rohrbach, Marcus and Yan, Shuicheng and Feng, Jiashi},
journal={arXiv preprint arXiv:1904.05049},
year={2019}
}
Third-party Implementations
- MXNet Implementation with imagenet training log by terrychenism
- Keras Implementation with cifar10 results by koshian2
Acknowledgement
- Thanks MXNet, Gluon-CV and TVM!
- Thanks @Ldpe2G for sharing the code for calculating the #FLOPs (
link) - Thanks Min Lin (Mila), Xin Zhao (Qihoo Inc.), Tao Wang (NUS) for helpful discussions on the code development.
Reference
[1] He K, et al "Identity Mappings in Deep Residual Networks".
[2] Christian S, et al "Rethinking the Inception Architecture for Computer Vision"
[3] Zhang H, et al. "mixup: Beyond empirical risk minimization.".
License
The code and the models are MIT licensed, as found in the LICENSE file.