TanayNarshana / Awasome-Pruning

Awasome Papers and Resources in Convolution Neural Network Pruning with Code.

Awasome Pruning

Awasome Resources in Deep Neural Network Pruning. This collection is prepared inspired by he-y/Awesome-Pruning.

Symbol	U	S	A	O
Details	Unstructured or Weight Pruning	Structured or Filter or Channel Pruning	Official or Author Implementation	Unofficial or 3rd Party Implementation

Conference

Year	Venue	Title	Type	Code
2023	ICLR	OTOv2: Automatic, Generic, User-Friendly	S	PyTorch[A]
2023	ICLR	How I Learned to Stop Worrying and Love Retraining	U	PyTorch[A]
2023	ICLR	Token Merging: Your ViT But Faster	U/S	PyTorch[A]
2023	ICLR	Revisiting Pruning at Initialization Through the Lens of Ramanujan Graphs	U	PyTorch[A] (soon...)
2023	ICLR	Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?	U
2023	ICLR	NTK-SAP: Improving neural network pruning by aligning training dynamics	U
2023	ICLR	DFPC: Data flow driven pruning of coupled channels without data	S	PyTorch[A]
2023	ICLR	TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning	S	PyTorch[A]
2023	ICLR	Pruning Deep Neural Networks from a Sparsity Perspective	U	PyTorch[A]
2023	ICLR	A Unified Framework of Soft Threshold Pruning	U	PyTorch[A]
2023	WACV	Calibrating Deep Neural Networks Using Explicit Regularisation and Dynamic Data Pruning	S
2023	WACV	Attend Who Is Weak: Pruning-Assisted Medical Image Localization Under Sophisticated and Implicit Imbalances	S
2023	ICASSP	WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks	S	PyTorch[A]
2022	CVPR	Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs	U
2022	CVPR	Revisiting Random Channel Pruning for Neural Network Compression	S	PyTorch[A] (soon...)
2022	CVPR	Fire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask Prediction	S	PyTorch[A]
2022	CVPR	When to Prune? A Policy towards Early Structural Pruning	S
2022	CVPR	Dreaming to Prune Image Deraining Networks	S
2022	ICLR	SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning	S
2022	ICLR	Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining	U	PyTorch[A]
2022	ICLR	Revisit Kernel Pruning with Lottery Regulated Grouped Convolutions	S	PyTorch[A]
2022	ICLR	Dual Lottery Ticket Hypothesis	U	PyTorch[A]
2022	NIPS	SAViT: Structure-Aware Vision Transformer Pruning via Collaborative Optimization	S	PyTorch[A](soon...)
2022	NIPS	Structural Pruning via Latency-Saliency Knapsack	S	PyTorch[A]
2022	ACCV	Filter Pruning via Automatic Pruning Rate Search⋆	S
2022	ACCV	Network Pruning via Feature Shift Minimization	S	PyTorch[A]
2022	ACCV	Lightweight Alpha Matting Network Using Distillation-Based Channel Pruning	S	PyTorch[A]
2022	ACCV	Adaptive FSP : Adaptive Architecture Search with Filter Shape Pruning	S
2022	ECCV	Soft Masking for Cost-Constrained Channel Pruning	S	PyTorch[A]
2022	WACV	Hessian-Aware Pruning and Optimal Neural Implant	S	PyTorch[A]
2022	WACV	PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression	S
2022	WACV	Channel Pruning via Lookahead Search Guided Reinforcement Learning	S
2022	WACV	EZCrop: Energy-Zoned Channels for Robust Output Pruning	S	PyTorch[A]
2022	ICIP	One-Cycle Pruning: Pruning Convnets With Tight Training Budget	U
2022	ICIP	RAPID: A Single Stage Pruning Framework	U
2022	ICIP	The Rise of the Lottery Heroes: Why Zero-Shot Pruning is Hard	U
2022	ICIP	Truncated Lottery Ticket for Deep Pruning	U
2022	ICIP	Which Metrics For Network Pruning: Final Accuracy? or Accuracy Drop?	S/U
2022	ISMSI	Structured Pruning with Automatic Pruning Rate Derivation for Image Processing Neural Networks	S
2021	ICLR	Neural Pruning via Growing Regularization	S	PyTorch[A]
2021	ICLR	Network Pruning That Matters: A Case Study on Retraining Variants	S	PyTorch[A]
2021	ICLR	Layer-adaptive Sparsity for the Magnitude-based Pruning	U	PyTorch[A]
2021	NIPS	Only Train Once: A One-Shot Neural Network Training And Pruning Framework	S	PyTorch[A]
2021	CVPR	NPAS: A Compiler-Aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration	S
2021	CVPR	Network Pruning via Performance Maximization	S
2021	CVPR	Convolutional Neural Network Pruning With Structural Redundancy Reduction*	S
2021	CVPR	Manifold Regularized Dynamic Network Pruning	S	PyTorch[A]
2021	CVPR	Joint-DetNAS: Upgrade Your Detector With NAS, Pruning and Dynamic Distillation	S
2021	ICCV	ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting	S
2021	ICCV	Achieving On-Mobile Real-Time Super-Resolution With Neural Architecture and Pruning Search	S
2021	ICCV	GDP: Stabilized Neural Network Pruning via Gates With Differentiable Polarization*	S
2021	WACV	Holistic Filter Pruning for Efficient Deep Neural Networks	S
2021	ICML	Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework	S
2021	ICML	Group Fisher Pruning for Practical Network Compression	S	PyTorch[A]
2020	CVPR	HRank: Filter Pruning using High-Rank Feature Map	S	PyTorch[A]
2020	CVPR	Towards efficient model compression via learned global ranking	S	PyTorch[A]
2020	CVPR	Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration	S
2020	CVPR	Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression	S	PyTorch[A]
2020	CVPR	APQ: Joint Search for Network Architecture, Pruning and Quantization Policy	S	PyTorch[A]
2020	ICLR	Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints	U
2020	MLSys	Shrinkbench: What is the State of Neural Network Pruning?		PyTorch[A]
2020	BMBS	Similarity Based Filter Pruning for Efficient Super-Resolution Models	S
2019	CVPR	Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration	S	PyTorch[A]
2019	CVPR	Variational Convolutional Neural Network Pruning	S
2019	CVPR	Towards Optimal Structured CNN Pruning via Generative Adversarial Learning	S	PyTorch[A]
2019	CVPR	Partial Order Pruning: For Best Speed/Accuracy Trade-Off in Neural Architecture Search	S	PyTorch[A]
2019	CVPR	Importance Estimation for Neural Network Pruning	S	PyTorch[A]
2019	ICLR	The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks	U	PyTorch[A]
2019	ICLR	SNIP: Single-shot Network Pruning based on Connection Sensitivity	U	Tensorflow[A]
2019	ICCV	MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning	S	PyTorch[A]
2019	ICCV	Accelerate CNN via Recursive Bayesian Pruning	S
2018	CVPR	PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning	S	PyTorch[A]
2018	CVPR	NISP: Pruning Networks Using Neuron Importance Score Propagation	S
2018	ICIP	Online Filter Clustering and Pruning for Efficient Convnets	S
2018	IJCAI	Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks	S	PyTorch[A]
2017	CVPR	Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning	S
2017	ICLR	Pruning Filters for Efficient ConvNets	S	PyTorch[O]
2017	ICCV	Channel Pruning for Accelerating Very Deep Neural Networks	S	PyTorch[A]
2017	ICCV	ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression	S	Caffe[A]
2017	ICCV	Learning Efficient Convolutional Networks Through Network Slimming	S	PyTorch[A]

Journal

Year	Journal	Title	Type	Code
2023	Neurocomputing	Filter pruning with uniqueness mechanism in the frequency domain for efficient neural networks	S
2023	IEEE Trans. PAMI	Performance-aware Approximation of Global Channel Pruning for Multitask CNNs	S	PyTorch[A]
2023	IEEE Trans. PAMI	Adaptive Search-and-Training for Robust and Efficient Network Pruning	S
2022	IEEE Trans. PAMI	Learning to Explore Distillability and Sparsability: A Joint Framework for Model Compression	S
2022	IEEE Trans. PAMI	1xN Pattern for Pruning Convolutional Neural Networks	S	PyTorch[A]
2022	IEEE Trans. NNLS	Filter Pruning by Switching to Neighboring CNNs With Good Attribute	S
2022	IEEE Trans. NNLS	Model Pruning Enables Efficient Federated Learning on Edge Devices	S
2022	IEEE Trans. NNLS	Network Pruning Using Adaptive Exemplar Filters	S	PyTorch[A]
2022	IEEE Trans. NNLS	Filter Sketch for Network Pruning	S	PyTorch[A]
2022	Neurocomputing	FPFS: Filter-level pruning via distance weight measuring filter similarity	S
2022	Neurocomputing	RUFP: Reinitializing unimportant filters for soft pruning	S
2022	Neural Netw	HRel: Filter pruning based on High Relevance between activation maps and class labels	S	PyTorch[A]*
2022	Comput. Intell. Neurosci.	Differentiable Network Pruning via Polarization of Probabilistic Channelwise Soft Masks	S
2022	J. Syst. Archit.	Optimizing deep neural networks on intelligent edge accelerators via flexible-rate filter pruning	S
2022	Appl. Sci.	Magnitude and Similarity Based Variable Rate Filter Pruning for Efficient Convolution Neural Networks	S	PyTorch[A]
2022	Sensors	Filter Pruning via Measuring Feature Map Information	S
2022	IEEE Access	Automated Filter Pruning Based on High-Dimensional Bayesian Optimization	S
2022	IEEE Signal Process. Lett.	A Low-Complexity Modified ThiNet Algorithm for Pruning Convolutional Neural Networks	S
2021	IEEE Trans. PAMI	Discrimination-Aware Network Pruning for Deep Model Compression	S	PyTorch[A]~
2020	IEEE Access	Filter Pruning Without Damaging Networks Capacity	S
2020	Electronics	Pruning Convolutional Neural Networks with an Attention Mechanism for Remote Sensing Image Classification	S

Survery Papers

Year	Venue	Title
2023	arVix	Structured Pruning for Deep Convolutional Neural Networks: A survey
2022	Electronics	A Survey on Efficient Convolutional Neural Networks and Hardware Acceleration
2022	I-SMAC	A Survey on Filter Pruning Techniques for Optimization of Deep Neural Networks
2021	JMLR	Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
2021	Neurocomputing	Pruning and quantization for deep neural network acceleration: A survey
2020	IEEE Access	Methods for Pruning Deep Neural Networks

Other

Year	Venue	Title	Code
2023	arVix	Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning	PyTorch[A](soon...)
2023	arVix	Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers
2022	ICML	Tutorial -- Sparsity in Deep Learning: Pruning and growth for efficient inference and training

Pruning Software/Toolbox

Year	Title	Type	Code
2022	FasterAI: Prune and Distill your models with FastAI and PyTorch	U	PyTorch[A]
2022	Simplify: A Python library for optimizing pruned neural networks		PyTorch[A]
2021	DepGraph: Towards Any Structural Pruning	S	PyTorch[A]
2021	PyTorchViz [A small package to create visualizations of PyTorch execution graphs]		PyTorch[A]
2020	What is the State of Neural Network Pruning?	S/U	PyTorch[A]
2019	Official PyTorch Pruning Tool	S/U	PyTorch[A]