Maojianzeng / Some-Attention-codes

注意力机制在计算机视觉领域的应用主要使用于捕捉图像上的respective field，而在自然语言处理领域中的应用主要使用于定位关键的token。这个项目收集了几种代码简洁的注意力机制，调用起来十分方便，可以轻松移植到自己的网络中，提升性能。

Some-Attention-codes

The application of attention mechanism in the field of computer vision is mainly used to capture the respective field on the image, while the application in the field of natural language processing is mainly used to locate the key token. This project collects several attention mechanisms with concise code, which are very convenient to call and can be easily ported to your own network to improve performance.

There are a total of 8 attention mechanisms, among which fcanet.py and layer.py are the codes of the same method, and the rest of the attention mechanisms have only one .py file.

Attached below are links to papers or open source projects for these attention mechanisms：

SE-Net： https://openaccess.thecvf.com/content_cvpr_2018/html/Hu_Squeeze-and-Excitation_Networks_CVPR_2018_paper.html

SK-Net：http://openaccess.thecvf.com/content_CVPR_2019/papers/Li_Selective_Kernel_Networks_CVPR_2019_paper.pdf

SPA-Net：https://ieeexplore.ieee.org/document/9102906

ECA-Net：https://openaccess.thecvf.com/content_CVPR_2020/papers/Wang_ECA-Net_Efficient_Channel_Attention_for_Deep_Convolutional_Neural_Networks_CVPR_2020_paper.pdf

CBAM：https://openaccess.thecvf.com/content_ECCV_2018/papers/Sanghyun_Woo_Convolutional_Block_Attention_ECCV_2018_paper.pdf

  https://github.com/Jongchan/attention-module/blob/master/MODELS/cbam.py

scSE：https://arxiv.org/abs/1803.02579

A2-Nets：https://papers.nips.cc/paper/7318-a2-nets-double-attention-networks.pdf