Refining activation downsampling with SoftPool

Abstract

Convolutional Neural Networks (CNNs) use pooling to decrease the size of activation maps. This process is crucial to locally achieve spatial invariance and to increase the receptive field of subsequent convolutions. Pooling operations should minimize the loss of information in the activation maps. At the same time, the computation and memory overhead should be limited. To meet these requirements, we propose SoftPool: a fast and efficient method that sums exponentially weighted activations. Compared to a range of other pooling methods, SoftPool retains more information in the downsampled activation maps. More refined downsampling leads to better classification accuracy. On ImageNet1K, for a range of popular CNN architectures, replacing the original pooling operations with SoftPool leads to consistent accuracy improvements in the order of 1-2%. We also test SoftPool on video datasets for action recognition. Again, replacing only the pooling layers consistently increases accuracy while computational load and memory remain limited. These favorable properties make SoftPool an excellent replacement for current pooling operations, including max-pool and average-pool.

[arXiv preprint ]

Image based pooling. Images are sub-sampled in both height and width by half.

Original
Soft Pool

Video based pooling. Videos are sub-sampled in time, height and width by half.

Original
Soft Pool

Dependencies

All parts of the code assume that torch is of version 1.4 or higher. There might be instability issues on previous versions.

! Disclaimer: This repository is heavily structurally influenced on Ziteng Gao's LIP repo https://github.com/sebgao/LIP

Installation

You can build the repo through the following commands:

$ git clone https://github.com/alexandrosstergiou/SoftPool.git
$ cd SoftPool-master/pytorch
$ make install
--- (optional) ---
$ make test

Usage

You can load any of the 1D, 2D or 3D variants after the installation with:

import softpool_cuda
from SoftPool import soft_pool1d, SoftPool1d
from SoftPool import soft_pool2d, SoftPool2d
from SoftPool import soft_pool3d, SoftPool3d

soft_poolxd: Is a functional interface for SoftPool.
SoftPoolxd: Is the class-based version which created an object that can be referenced later in the code.

ImageNet models

ImageNet weight can be downloaded from the following links:

Network	link
ResNet-18	link
ResNet-34	link
ResNet-50	link
ResNet-101	link
ResNet-152	link
DenseNet-121	link
DenseNet-161	link
DenseNet-169	link
ResNeXt-50_32x4d	link
ResNeXt-101_32x4d	link
wide-ResNet50	link

Citation

@article{ stergiou2021refining,
  title={Refining activation downsampling with SoftPool},
  author={Stergiou, Alexandros, Poppe, Ronald and Kalliatakis Grigorios},
  journal={arXiv preprint arXiv:2101.00440},
  year={2021}
}

Licence

MIT

Additional resources

A great project is Ren Tianhe's pytorh-pooling repo for overviewing different pooling strategies.

yisun98 / SoftPool