Awesome Pruning

A curated list of neural network pruning and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers and Awesome-NAS.

Please feel free to pull requests or open an issue to add papers.

Type of Pruning
2022 Venues
2021 Venues
2020 Venues
2019 Venues
2018 Venues
2017 Venues
2016 Venues
2015 Venues

Type of Pruning

Type	`F`	`W`	`Other`
Explanation	Filter pruning	Weight pruning	other types

2022

Title	Venue	Type	Code
FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server	IJCAI	`F`	-
On the Channel Pruning using Graph Convolution Network for Convolutional Neural Network Acceleration	IJCAI	`F`	-
Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization	IJCAI	`F`	-
Neural Network Pruning by Cooperative Coevolution	IJCAI	`F`	-
Recent Advances on Neural Network Pruning at Initialization	IJCAI	`W`	PyTorch(Author)
Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness	ICML	`F`	PyTorch(Author)
SPDY: Accurate Pruning with Speedup Guarantees	ICML	`W`	PyTorch(Author)
Sparse Double Descent: Where Network Pruning Aggravates Overfitting	ICML	`W`	PyTorch(Author)
PAC-Net: A Model Pruning Approach to Inductive Transfer Learning	ICML	`Other`	-
Neural Network Pruning Denoises the Features and Makes Local Connectivity Emerge in Visual Tasks	ICML	`Other`	PyTorch(Author)
Winning the Lottery Ahead of Time: Efficient Early Network Pruning	ICML	`F`	-
Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning	ICML	`F`	PyTorch(Author)
The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks	ICML	`W`	PyTorch(Author)
DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks	ICML	`Other`	PyTorch(Author)
Fast Lossless Neural Compression with Integer-Only Discrete Flows	ICML	`F`	PyTorch(Author)
Fire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask PredictionFire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask Prediction	CVPR	`F`	-
Revisiting Random Channel Pruning for Neural Network Compression	CVPR	`F`	PyTorch(Author)(Releasing)
Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs	CVPR	`W`	-
When To Prune? A Policy Towards Early Structural Pruning	CVPR	`F`	-
DiSparse: Disentangled Sparsification for Multitask Model Compression	CVPR	`Other`	PyTorch(Author)(Releasing)
Contrastive Dual Gating: Learning Sparse Features With Contrastive Learning	CVPR	`WF`	-
Learning Bayesian Sparse Networks With Full Experience Replay for Continual Learning	CVPR	`F`	-
Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network	CVPR	`W`	-
DECORE: Deep Compression With Reinforcement Learning	CVPR	`F`	-
CHEX: CHannel EXploration for CNN Model Compression	CVPR	`F`	-
Compressing Models With Few Samples: Mimicking Then Replacing	CVPR	`F`	PyTorch(Author)(Releasing)
Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining	ICLR (Spotlight)	`W`	PyTorch(Author)
On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning	ICLR (Spotlight)	`W`	-
SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning	ICLR (Spotlight)	`F`	PyTorch(Author)(Releasing)
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models	ICLR (Spotlight)	`F`	PyTorch(Author)
Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients	ICLR	`WF`	PyTorch(Author)
An Operator Theoretic View On Pruning Deep Neural Networks	ICLR	`W`	-
Revisit Kernel Pruning with Lottery Regulated Grouped Convolutions	ICLR	`F`	PyTorch(Author)
Effective Model Sparsification by Scheduled Grow-and-Prune Methods	ICLR	`W`	PyTorch(Author)
The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training	ICLR	`Other`	PyTorch(Author)
Learning Efficient Image Super-Resolution Networks via Structure-Regularized Pruning	ICLR	`F`	PyTorch(Author)
Signing the Supermask: Keep, Hide, Invert	ICLR	`W`	-
Plant 'n' Seek: Can You Find the Winning Ticket?	ICLR	`F`	PyTorch(Author)
Proving the Lottery Ticket Hypothesis for Convolutional Neural Networks	ICLR	`F`	PyTorch(Author)
On the Existence of Universal Lottery Tickets	ICLR	`F`	PyTorch(Author)
Training Structured Neural Networks Through Manifold Identification and Variance Reduction	ICLR	`F`	PyTorch(Author)
How many degrees of freedom do we need to train deep networks: a loss landscape perspective	ICLR	`W`	PyTorch(Author)
Dual Lottery Ticket Hypothesis	ICLR	`W`	PyTorch(Author)
Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently	ICLR	`W`	PyTorch(Author)
Sparsity Winning Twice: Better Robust Generalization from More Efficient Training	ICLR	`W`	PyTorch(Author)
Prior Gradient Mask Guided Pruning-Aware Fine-Tuning	AAAI	`F`	-
Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition	AAAI	`Other`	-
Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks	AAAI	`W`	-

2021

Title	Venue	Type	Code
Validating the Lottery Ticket Hypothesis with Inertial Manifold Theory	NeurIPS	`W`	-
The Elastic Lottery Ticket Hypothesis	NeurIPS	`W`	PyTorch(Author)
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?	NeurIPS	`W`	PyTorch(Author)
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks	NeurIPS	`W`	-
You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership	NeurIPS	`W`	PyTorch(Author)
Pruning Randomly Initialized Neural Networks with Iterative Randomization	NeurIPS	`W`	PyTorch(Author)
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration	NeurIPS	`W`	PyTorch(Author)
Rethinking the Pruning Criteria for Convolutional Neural Network	NeurIPS	`F`	-
Only Train Once: A One-Shot Neural Network Training And Pruning Framework	NeurIPS	`F`	PyTorch(Author)
CHIP: CHannel Independence-based Pruning for Compact Neural Networks	NeurIPS	`F`	PyTorch(Author)
Sparse Flows: Pruning Continuous-depth Models	NeurIPS	`WF`	PyTorch(Author)
A Winning Hand: Compressing Deep Networks Can Improve Out-of-Distribution Robustness	NeurIPS	`W`	PyTorch(Author)
Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition	NeurIPS	`F`	PyTorch(Author)
AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks	NeurIPS	`W`	PyTorch(Author)
RED : Looking for Redundancies for Data-FreeStructured Compression of Deep Neural Networks	NeurIPS	`F`	-
Scaling Up Exact Neural Network Compression by ReLU Stability	NeurIPS	`F`	PyTorch(Author)
Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme	NeurIPS	`F`	PyTorch(Author)
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks	NeurIPS	`Other`	PyTorch(Author)
ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting	ICCV	`F`	PyTorch(Author)
Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search	ICCV	`F`	-
GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization	ICCV	`F`	-
Auto Graph Encoder-Decoder for Neural Network Pruning	ICCV	`F`	-
Sub-Bit Neural Networks: Learning To Compress and Accelerate Binary Neural Networks	ICCV	`Other`	PyTorch(Author)
Exploration and Estimation for Model Compression	ICCV	`F`	-
A Probabilistic Approach to Neural Network Pruning	ICML	`F`	-
Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework	ICML	`F`	-
Group Fisher Pruning for Practical Network Compression	ICML	`F`	PyTorch(Author)
On the Predictability of Pruning Across Scales	ICML	`W`	-
Towards Compact CNNs via Collaborative Compression	CVPR	`F`	PyTorch(Author)
Content-Aware GAN Compression	CVPR	`F`	PyTorch(Author)
Permute, Quantize, and Fine-tune: Efficient Compression of Neural Networks	CVPR	`F`	PyTorch(Author)
NPAS: A Compiler-aware Framework of Unified Network Pruning andArchitecture Search for Beyond Real-Time Mobile Acceleration	CVPR	`F`	-
Network Pruning via Performance Maximization	CVPR	`F`	-
Convolutional Neural Network Pruning with Structural Redundancy Reduction	CVPR	`F`	-
Manifold Regularized Dynamic Network Pruning	CVPR	`F`	-
Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation	CVPR	`FO`	-
A Gradient Flow Framework For Analyzing Network Pruning	ICLR	`F`	PyTorch(Author)
Neural Pruning via Growing Regularization	ICLR	`F`	PyTorch(Author)
ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximations	ICLR	`F`	PyTorch(Author)
Network Pruning That Matters: A Case Study on Retraining Variants	ICLR	`F`	PyTorch(Author)
Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network	ICLR	`W`	PyTorch(Author)
Layer-adaptive Sparsity for the Magnitude-based Pruning	ICLR	`W`	PyTorch(Author)
Pruning Neural Networks at Initialization: Why Are We Missing the Mark?	ICLR	`W`	-
Robust Pruning at Initialization	ICLR	`W`	-

2020

Title	Venue	Type	Code
HYDRA: Pruning Adversarially Robust Neural Networks	NeurIPS	`W`	PyTorch(Author)
Logarithmic Pruning is All You Need	NeurIPS	`W`	-
Directional Pruning of Deep Neural Networks	NeurIPS	`W`	-
Movement Pruning: Adaptive Sparsity by Fine-Tuning	NeurIPS	`W`	PyTorch(Author)
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot	NeurIPS	`W`	PyTorch(Author)
Neuron Merging: Compensating for Pruned Neurons	NeurIPS	`F`	PyTorch(Author)
Neuron-level Structured Pruning using Polarization Regularizer	NeurIPS	`F`	PyTorch(Author)
SCOP: Scientific Control for Reliable Neural Network Pruning	NeurIPS	`F`	PyTorch(Author)
Storage Efficient and Dynamic Flexible Runtime Channel Pruning via Deep Reinforcement Learning	NeurIPS	`F`	-
The Generalization-Stability Tradeoff In Neural Network Pruning	NeurIPS	`F`	PyTorch(Author)
Pruning Filter in Filter	NeurIPS	`Other`	PyTorch(Author)
Position-based Scaled Gradient for Model Quantization and Pruning	NeurIPS	`Other`	PyTorch(Author)
Bayesian Bits: Unifying Quantization and Pruning	NeurIPS	`Other`	-
Pruning neural networks without any data by iteratively conserving synaptic flow	NeurIPS	`Other`	PyTorch(Author)
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning	ECCV (Oral)	`F`	PyTorch(Author)
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation	ECCV	`F`	-
DHP: Differentiable Meta Pruning via HyperNetworks	ECCV	`F`	PyTorch(Author)
Meta-Learning with Network Pruning	ECCV	`W`	-
Accelerating CNN Training by Pruning Activation Gradients	ECCV	`W`	-
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search	ECCV	`Other`	-
Differentiable Joint Pruning and Quantization for Hardware Efficiency	ECCV	`Other`	-
Channel Pruning via Automatic Structure Search	IJCAI	`F`	PyTorch(Author)
Adversarial Neural Pruning with Latent Vulnerability Suppression	ICML	`W`	-
Proving the Lottery Ticket Hypothesis: Pruning is All You Need	ICML	`W`	-
Soft Threshold Weight Reparameterization for Learnable Sparsity	ICML	`WF`	Pytorch(Author)
Network Pruning by Greedy Subnetwork Selection	ICML	`F`	-
Operation-Aware Soft Channel Pruning using Differentiable Masks	ICML	`F`	-
DropNet: Reducing Neural Network Complexity via Iterative Pruning	ICML	`F`	-
Towards Efficient Model Compression via Learned Global Ranking	CVPR (Oral)	`F`	Pytorch(Author)
HRank: Filter Pruning using High-Rank Feature Map	CVPR (Oral)	`F`	Pytorch(Author)
Neural Network Pruning with Residual-Connections and Limited-Data	CVPR (Oral)	`F`	-
Multi-Dimensional Pruning: A Unified Framework for Model Compression	CVPR (Oral)	`WF`	-
DMCP: Differentiable Markov Channel Pruning for Neural Networks	CVPR (Oral)	`F`	TensorFlow(Author)
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression	CVPR	`F`	PyTorch(Author)
Few Sample Knowledge Distillation for Efficient Network Compression	CVPR	`F`	-
Discrete Model Compression With Resource Constraint for Deep Neural Networks	CVPR	`F`	-
Structured Compression by Weight Encryption for Unstructured Pruning and Quantization	CVPR	`W`	-
Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration	CVPR	`F`	-
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy	CVPR	`F`	-
Comparing Rewinding and Fine-tuning in Neural Network Pruning	ICLR (Oral)	`WF`	TensorFlow(Author)
A Signal Propagation Perspective for Pruning Neural Networks at Initialization	ICLR (Spotlight)	`W`	-
ProxSGD: Training Structured Neural Networks under Regularization and Constraints	ICLR	`W`	TF+PT(Author)
One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation	ICLR	`W`	-
Lookahead: A Far-sighted Alternative of Magnitude-based Pruning	ICLR	`W`	PyTorch(Author)
Dynamic Model Pruning with Feedback	ICLR	`WF`	-
Provable Filter Pruning for Efficient Neural Networks	ICLR	`F`	-
Data-Independent Neural Pruning via Coresets	ICLR	`W`	-
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates	AAAI	`F`	-
DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks	AAAI	`Other`	-
Pruning from Scratch	AAAI	`Other`	-
Reborn filters: Pruning convolutional neural networks with limited data	AAAI	`F`	-

2019

Title	Venue	Type	Code
Network Pruning via Transformable Architecture Search	NeurIPS	`F`	PyTorch(Author)
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks	NeurIPS	`F`	PyTorch(Author)
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask	NeurIPS	`W`	TensorFlow(Author)
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers	NeurIPS	`W`	-
Global Sparse Momentum SGD for Pruning Very Deep Neural Networks	NeurIPS	`W`	PyTorch(Author)
AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters	NeurIPS	`W`	-
Model Compression with Adversarial Robustness: A Unified Optimization Framework	NeurIPS	`Other`	PyTorch(Author)
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning	ICCV	`F`	PyTorch(Author)
Accelerate CNN via Recursive Bayesian Pruning	ICCV	`F`	-
Adversarial Robustness vs Model Compression, or Both?	ICCV	`W`	PyTorch(Author)
Learning Filter Basis for Convolutional Neural Network Compression	ICCV	`Other`	-
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration	CVPR (Oral)	`F`	PyTorch(Author)
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning	CVPR	`F`	PyTorch(Author)
Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure	CVPR	`F`	PyTorch(Author)
On Implicit Filter Level Sparsity in Convolutional Neural Networks, Extension1, Extension2	CVPR	`F`	PyTorch(Author)
Structured Pruning of Neural Networks with Budget-Aware Regularization	CVPR	`F`	-
Importance Estimation for Neural Network Pruning	CVPR	`F`	PyTorch(Author)
OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks	CVPR	`F`	-
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search	CVPR	`Other`	TensorFlow(Author)
Variational Convolutional Neural Network Pruning	CVPR	-	-
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks	ICLR (Best)	`W`	TensorFlow(Author)
Rethinking the Value of Network Pruning	ICLR	`F`	PyTorch(Author)
Dynamic Channel Pruning: Feature Boosting and Suppression	ICLR	`F`	TensorFlow(Author)
SNIP: Single-shot Network Pruning based on Connection Sensitivity	ICLR	`W`	TensorFLow(Author)
Dynamic Sparse Graph for Efficient Deep Learning	ICLR	`F`	CUDA(3rd)
Collaborative Channel Pruning for Deep Networks	ICML	`F`	-
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization github	ICML	`F`	-
EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis	ICML	`F`	PyTorch(Author)
COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning	IJCAI	`F`	Tensorflow(Author)

2018

Title	Venue	Type	Code
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers	ICLR	`F`	TensorFlow(Author), PyTorch(3rd)
To prune, or not to prune: exploring the efficacy of pruning for model compression	ICLR	`W`	-
Discrimination-aware Channel Pruning for Deep Neural Networks	NeurIPS	`F`	TensorFlow(Author)
Frequency-Domain Dynamic Pruning for Convolutional Neural Networks	NeurIPS	`W`	-
Learning Sparse Neural Networks via Sensitivity-Driven Regularization	NeurIPS	`WF`	-
Amc: Automl for model compression and acceleration on mobile devices	ECCV	`F`	TensorFlow(3rd)
Data-Driven Sparse Structure Selection for Deep Neural Networks	ECCV	`F`	MXNet(Author)
Coreset-Based Neural Network Compression	ECCV	`F`	PyTorch(Author)
Constraint-Aware Deep Neural Network Compression	ECCV	`W`	SkimCaffe(Author)
A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers	ECCV	`W`	Caffe(Author)
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning	CVPR	`F`	PyTorch(Author)
NISP: Pruning Networks using Neuron Importance Score Propagation	CVPR	`F`	-
CLIP-Q: Deep Network Compression Learning by In-Parallel Pruning-Quantization	CVPR	`W`	-
“Learning-Compression” Algorithms for Neural Net Pruning	CVPR	`W`	-
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks	IJCAI	`F`	PyTorch(Author)
Accelerating Convolutional Networks via Global & Dynamic Filter Pruning	IJCAI	`F`	-

2017

Title	Venue	Type	Code
Pruning Filters for Efficient ConvNets	ICLR	`F`	PyTorch(3rd)
Pruning Convolutional Neural Networks for Resource Efficient Inference	ICLR	`F`	TensorFlow(3rd)
Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee	NeurIPS	`W`	TensorFlow(Author)
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon	NeurIPS	`W`	PyTorch(Author)
Runtime Neural Pruning	NeurIPS	`F`	-
Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning	CVPR	`W`	-
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression	ICCV	`F`	Caffe(Author), PyTorch(3rd)
Channel pruning for accelerating very deep neural networks	ICCV	`F`	Caffe(Author)
Learning Efficient Convolutional Networks Through Network Slimming	ICCV	`F`	PyTorch(Author)

2016

Title	Venue	Type	Code
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding	ICLR (Best)	`W`	Caffe(Author)
Dynamic Network Surgery for Efficient DNNs	NeurIPS	`W`	Caffe(Author)