jason-li-831202 / Flexible-Conv-Module

⭐⭐⭐ Pytorch implementation of Attentiom, Backbone, ViT, MLP, Re-parameter, Convolution, very flexible module combination.

Flexible-Conv-Module

The Flexible-Conv-Module library is a flexible toolbox of a series of CV algorithms based on PyTorch. Used to combine different modules to build different networks.

➤ Contents

1. Requirements

2. Main Components

   • Conv Basic Series
   • Conv Block Series
   • Conv ReceptionField Series
   • Conv Backbone Series
   • ReParameter Series
   • Vision Transformer Series
   • Attention Series

➤ Requirements🚀

• Install :

  The requirements.txt file should list all Python libraries that your notebooks depend on, and they will be installed using:

  pip install -r requirements.txt
  

➤ Main Components🚀

Conv Module Components

Basic	Block	ReParameter	ReceptionField	Backbone
CondConv DoConv DynamicConv PConv PsConv PyConv	BsConv DwsConv GhostConv GnConv MixConv ScConv	AC Block DiverseBranch Block Mobileone Block RepLK Block RepMLP Block RepVGG Block	SPP Module SPPF Module ASPP Module SPPCSPC Module SPPFCSPC Module RFB Module	BottleNeck Block Resnet Block Resnext Block VarGroup Block VoV Block HGnet Block Pelee Block Mobilenext Block EPSA Block CGNet Block

Transformer Module Componens

Attention	ViT
CBAModule coordAttModule EAModule ECAModule GAModule LKAModule SEModule shuffleAttModule simAModule SKModule ULSAModule PSAModule	ConvNext Block EdgeViT Block Hor Block MobileVit Block PVTv1 Block PVTv2 Block TnT Block WaveMLP Block STViT Block

👉 Conv Basic Series

• 1 - CondConv: Conditionally Parameterized Convolutions for Efficient Inference

  • NeurIPS 2019
  • Usage : CondConv2d

• 2 - DO-Conv: Depthwise Over-parameterized Convolutional Layer

  • TIP 2022
  • Usage : DoConv2d

• 3 - Dynamic Convolution: Attention over Convolution Kernels

  • CVPR 2020
  • Usage : DynamicConv2d

• 4 - PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer

  • ECCV 2020
  • Usage : PSConv2d

• 5 - Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition

  • CVPR 2020
  • Usage : PyConv2d

• 6 - Run, Don’t Walk: Chasing Higher FLOPS for Faster Neural Networks

  • CVPR 2023
  • Usage : PConv2d

👉 Conv Block Series

• 1 - Blueprint Separable Residual Network for Efficient Image Super-Resolution

  • CVPR 2022
  • Usage : BlueprintSeparableUConv2d, BlueprintSeparableSConv2d

• 2 - PP-LCNet: A Lightweight CPU Convolutional Neural Network

  • CVPR 2021
  • Improve by DW Conv
  • Usage : DepthwiseSeparableConv2d

• 3.1 - GhostNet: More Features from Cheap Operations

  • CVPR 2020
  • Usage : GhostConv2d

• 3.2 - GhostNetV2: Enhance Cheap Operation with Long-Range Attention

  • NeurIPS 2022
  • Usage : Ghostv2Conv2d

• 4 - HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions

  • NeurIPS 2022
  • Usage : RecursiveGatedConv2d

• 5 - MixConv: Mixed Depthwise Convolutional Kernels

  • BMVC 2019
  • Usage : MixedDepthwiseConv2d

• 6 - SCConv: Spatial and Channel Reconstruction Convolution for Feature Redundancy

  • CVPR 2023
  • Usage : SCReconstructConv2d

👉 Conv ReceptionField Series

• 1 - Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

  • CVPR 2015
  • Usage : SPP, SPPBlock

• 2.1 - YOLOv5

  • Spatial Pyramid Pooling - Fast (SPPF) layer for YOLOv5 by Glenn Jocher
  • Usage : SPPF, SPPFBlock

• 2.2 - YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications

  • CVPR 2022
  • Usage : SimSPPFBlock

• 3.1 - DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

  • CVPR 2017
  • Usage : ASPPv2

• 3.2 - Rethinking Atrous Convolution for Semantic Image Segmentation

  • CVPR 2017
  • Usage : ASPPv3, ASPPv3Block

• 4 - Receptive Field Block Net for Accurate and Fast Object Detection

  • ECCV 2018
  • Usage : RFBBlock, RFBsBlock

• 5 - YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors

  • CVPR 2022
  • Usage : SPPCSPCBlock

• 6 - YOLOv6 v3.0: A Full-Scale Reloading

  • CVPR 2023
  • Usage : SPPFCSPCBlock

👉 Conv Backbone Series

• 1 - Deep Residual Learning for Image Recognition

  • CVPR 2015
  • Usage : ResnetUnit, ResnetStage

• 2 - Aggregated Residual Transformations for Deep Neural Networks

  • CVPR 2017
  • Usage : ResnextUnit, ResnextStage

• 3 - VarGFaceNet: An Efficient Variable Group Convolutional Neural Network for Lightweight Face Recognition

  • ICCV 2019
  • Usage : VarGroupConv, VarGroupBlock, VarGroupBlock_DownSampling, VarGroupStage

• 4 - An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection

  • CVPR 2019
  • Usage : OSAUnit, DWsOSAUnit, OSAStage

• 5 - High Performance GPU Net

  • Paddlepaddle
  • Usage : "HGUnit, HGStage

• 6 - Pelee: A Real-Time Object Detection System on Mobile Devices

  • CVPR 2019
  • Usage : PeleeUnit, PeleeStage

• 7 - Rethinking Bottleneck Structure for Efficient Mobile Network Design

  • ECCV 2020
  • Usage : MobilenextStage

• 8 - EPSANet: An Efficient Pyramid Squeeze Attention Block on Convolutional Neural Network

  • CVPR 2021
  • Usage : EPSAUnit, EPSAStage

• 9 - CGNet: A Light-weight Context Guided Network for Semantic Segmentation

  • CVPR 2019
  • Usage : ContextGuidedUnit, ContextGuidedStage

👉 ReParameter Series

• 1 - ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric

  • ICCV 2019
  • Usage : ACBlock

• 2 - Diverse Branch Block: Building a Convolution as an Inception-like Unit

  • CVPR 2021
  • Usage : DBBlock

• 3 - RepVGG: Making VGG-style ConvNets Great Again

  • CVPR 2021
  • Usage : RepVGGBlock, RepVGGStage

• 4 - MobileOne: An Improved One millisecond Mobile Backbone

  • CVPR 2022
  • Usage : MobileOneBlock, MobileOneUnit, MobileOneStage

• 5 - RepMLPNet: Hierarchical Vision MLP with Re-parameterized Locality

  • CVPR 2022
  • Usage : RepMLPBlock, RepMLPUnit

• 6 - Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs

  • CVPR 2022
  • Usage : RepLKBlock, RepLKUnit, RepLKStage

👉 Vision Transformer Series

• 1 - Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions

  • ICCV 2021
  • Usage : PVTv1Unit, PVTv1Stage

• 2 - Transformer in Transformer

  • NeurIPS 2021
  • Usage : TnTUnit

• 3 - MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer

  • ICLR 2021
  • Usage : MobileViTUnit, MobileViTStage

• 4 - PVT v2: Improved Baselines with Pyramid Vision Transformer

  • CVMJ 2022
  • Usage : PVTv2Unit, PVTv2Stage

• 5 - An Image Patch is a Wave: Quantum Inspired Vision MLP

  • CVPR 2022
  • Usage : WaveUnit, WaveStage

• 6 - HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions

  • NeurIPS 2022
  • Usage : HorUnit, HorStage

• 7 - EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers

  • ECCV 2022
  • Usage : EdgeViTUnit, EdgeViTStage

• 8 - A ConvNet for the 2020s

  • CVPR 2022
  • Usage : ConvNeXtUnit, ConvNeXtStage

• 9 - Vision Transformer with Super Token Sampling

  • CVPR 2023
  • Usage : STViTUnit, STViTStage

👉 Attention Series

• 1 - CBAM: Convolutional Block Attention Module

  • ECCV 2018
  • Usage : CAMBlock_FC, CAMBlock_Conv, SAMBlock, CBAMBlock

• 2 - Squeeze-and-Excitation Networks

  • CVPR 2018
  • Usage : SEBlock, SEBlock_Conv, ESEBlock_Conv

• 3 - Selective Kernel Networks

  • CVPR 2019
  • Usage : SKBlock

• 4 - ULSAM: Ultra-Lightweight Subspace Attention Module for Compact Convolutional Neural Networks

  • CVPR 2020
  • Usage : ULSAMBlock

• 5 - ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

  • CVPR 2020
  • Usage : ECABlock

• 6 - Coordinate Attention for Efficient Mobile Network Design

  • CVPR 2021
  • Usage : CoordAttBlock

• 7 - Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks

  • CVPR 2021
  • Usage : EABlock

• 8 - Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions

  • CVPR 2021
  • Usage : GAMBlock

• 9 - SA-NET: SHUFFLE ATTENTION FOR DEEP CONVOLUTIONAL NEURAL NETWORKS

  • ICASSP 2021
  • Usage : ShuffleAttBlock

• 10 - EPSANet: An Efficient Pyramid Squeeze Attention Block on Convolutional Neural Network

  • CVPR 2021
  • Usage : PSABlock

• 11 - Simple Attention Module based Speaker Verification with Iterative noisy label detection

  • ICASSP 2022
  • Usage : simAMBlock

• 12 - Beyond Self-Attention: Deformable Large Kernel Attention for Medical Image Segmentation

  • CVPR 2023
  • Usage : LKAConv, deformable_LKAConv, LKABlock