network-compression

There are 3 repositories under network-compression topic.

datawhalechina / leedl-tutorial
《李宏毅深度学习教程》（李宏毅老师推荐👍），PDF下载地址：https://github.com/datawhalechina/leedl-tutorial/releases
bert chatgpt cnn deep-learning diffusion gan leedl-tutorial machine-learning network-compression pruning reinforcement-learning rnn self-attention transfer-learning transformer tutorial
Language:Jupyter Notebook 9601
IntelLabs / distiller
Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller
pytorch pruning quantization pruning-structures jupyter-notebook network-compression deep-neural-networks regularization group-lasso distillation early-exit truncated-svd onnx automl-for-compression
Language:Jupyter Notebook 4302
quic / aimet
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
auto-ml compression deep-learning deep-neural-networks machine-learning network-compression network-quantization open-source opensource pruning quantization
Language:Python 1889
clovaai / overhaul-distillation
Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)
knowledge-distillation knowledge-transfer teacher-student network-compression iccv2019
Language:Python 409
sony / model_optimization
Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.
deep-learning quantization network-compression network-quantization deep-neural-networks machine-learning edge-ai neural-network optimizer ptq pytorch qat tensorflow
Language:Python 257
A-suozhang / awesome-quantization-and-fixed-point-training
Neural Network Quantization & Low-Bit Fixed Point Training For Hardware-Friendly Algorithm Design
fixed-point network-compression quantization
150
jshilong / FisherPruning
Group Fisher Pruning for Practical Network Compression(ICML2021)
network-compression pruning
Language:Python 149
uber-research / permute-quantize-finetune
Using ideas from product quantization for state-of-the-art neural network compression.
deep-learning network-compression vector-quantization
Language:Python 143
bhheo / AB_distillation
Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons (AAAI 2019)
knowledge-distillation knowledge-transfer network-compression teacher-student-learning transfer-learning
Language:Python 103
musco-ai / musco-pytorch
MUSCO: MUlti-Stage COmpression of neural networks
pytorch network-compression tensor-decomposition tucker cp-decomposition truncated-svd vbmf low-rank deep-neural-networks network-acceleration model-compression model-acceleration
Language:Jupyter Notebook 73
bhheo / BSS_distillation
Knowledge Distillation with Adversarial Samples Supporting Decision Boundary (AAAI 2019)
adversarial-attacks adversarial-samples image-classification knowledge-distillation network-compression teacher-student-learning
Language:Python 69
ofsoundof / group_sparsity
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression. CVPR2020.
network-compression filter-pruning low-rank-approximation group-sparsity proximal-gradient-descent
Language:Python 59
ofsoundof / dhp
This is the official implementation of "DHP: Differentiable Meta Pruning via HyperNetworks".
meta-learning network-compression pruning hypernetworks differentiable
Language:Python 57
ofsoundof / learning_filter_basis
Pytorch implemenation of "Learning Filter Basis for Convolutional Neural Network Compression" ICCV2019
image-classification low-rank-approximation network-compression super-resolution
Language:Python 16
Zhengyu-Li / Deep-Network-Compression-based-on-Student-Teacher-Network-
Deep Neural Network Compression based on Student-Teacher Network
network-compression deeplearning densenet teacher-student multi-gpu tensorflow
Language:Python 14
cambridge-mlg / arch_uncert
Code for "Variational Depth Search in ResNets" (https://arxiv.org/abs/2002.02797)
bayesian-inference expectation-maximization neural-architecture-search bayesian-neural-networks bayesian-deep-learning variational-inference reproducible-research reproducible-paper learnt-depth residual-networks resnets network-depths mnist fashion-mnist svhn uncertainty sparse-neural-networks network-compression
Language:Jupyter Notebook 9
yukaikw / Machine-Learning
李宏毅教授 ML 2020 機器學習課程筆記 & 實作
machine-learning deep-learning classification regression cnn rnn explainable-ai adversarial-attacks network-compression unsupervised-learning anomaly-detection gan transfer-learning
Language:Python 9
musco-ai / musco-tf
MUSCO: Multi-Stage COmpression of neural networks
musco tensorflow network-compression tensor-decomposition tucker cp-decomposition truncated-svd vbmf low-rank-approximation deep-neural-networks cnn-compresion cnn-acceleration
Language:Python 8
pvtien96 / CORING
:ring: Efficient tensor-based filter pruning
filter-pruning network-compression structured-pruning tensor-decomposition tinyml hosvd tucker-decomposition network-pruning
Language:Jupyter Notebook 8
malena1906 / Pruning-Weights-with-Biobjective-Optimization-Keras
Overparameterization and overfitting are common concerns when designing and training deep neural networks. Network pruning is an effective strategy used to reduce or limit the network complexity, but often suffers from time and computational intensive procedures to identify the most important connections and best performing hyperparameters. We suggest a pruning strategy which is completely integrated in the training process and which requires only marginal extra computational cost. The method relies on unstructured weight pruning which is re-interpreted in a multiobjective learning approach. A batchwise Pruning strategy is selected to be compared using different optimization methods, of which one is a multiobjective optimization algorithm. As it takes over the choice of the weighting of the objective functions, it has a great advantage in terms of reducing the time consuming hyperparameter search each neural network training suffers from. Without any a priori training, post training, or parameter fine tuning we achieve highly reductions of the dense layers of two commonly used convolution neural networks (CNNs) resulting in only a marginal loss of performance. Our results empirically demonstrate that dense layers are overparameterized as with reducing up to 98 % of its edges they provide almost the same results. We contradict the theory that retraining after pruning neural networks is of great importance and opens new insights into the usage of multiobjective optimization techniques in machine learning algorithms in a Keras framework. The Stochastic Multi Gradient Descent Algorithm implementation in Python3 is for usage with Keras and adopted from paper of S. Liu and L. N. Vicente: "The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning". It is combined with weight pruning strategies to reduce network complexity and inference time.
unstructured-weight-pruning multiobjective-learning soft-pruning sparsity automl network-compression stochastic-multi-gradient l1-regularization dichotomic-serach dense-layers keras tensorflow image-classification machine-learning ai python3
Language:Python 7
yuxwind / ExactCompression
[NeurIPS 2021] Official PyTorch Code of Scaling Up Exact Neural Network Compression by ReLU Stability
lossless-compression-algorithm network-compression
Language:Python 7
kartikgupta-at-anu / md-bnn
Code implementation of our AISTATS'21 paper "Mirror Descent View for Neural Network Quantization"
binarized-neural-networks mirror-descent network-compression quantized-neural-networks
Language:Python 6
Yifan122 / NetworkCompress
pruning
network-compression pruning quantization
Language:Python 5
IU-SAIGE / sparse_mle
2020 INTERSPEECH, "Sparse Mixture of Local Experts for Efficient Speech Enhancement".
speech-enhancement mixture-of-experts network-compression
Language:Python 4
quic / aimet-pages
AIMET GitHub pages documentation
quantization deep-learning compression open-source machine-learning pruning auto-ml network-compression deep-neural-networks network-quantization opensource
Language:HTML 4
MortalHappiness / ML2019SPRING
Homework for Machine Learning (2019, Spring) at NTU
machine-learning deep-learning linear-regression neural-network dnn cnn rnn keras pytorch unsupervised-learning network-compression model-attack clustering pca
Language:Python 3
shaharpit809 / Deep-Learning-Models
This repository consists of application of Deep Learning Models like DNN, CNN (1D and 2D), RNN (LSTM and GRU) and Variational Autoencoders written from scratch in tensorflow.
convolutional-neural-network deep-neural-network lstm network-compression speaker-verification speech-recognition variational-autoencoders
Language:Jupyter Notebook 2
maxblumental / network-compression
neural-networks svd compression network-compression android tflite
Language:Jupyter Notebook 1
Hulalazz / Deep-Compression-AlexNet
Deep Compression on AlexNet
network-compression
Language:Python
Hulalazz / Embedded-Neural-Network
collection of works aiming at reducing model sizes or the ASIC/FPGA accelerator for machine learning
network-compression network-pruning accelerated-network

network-compression

datawhalechina / leedl-tutorial

IntelLabs / distiller

quic / aimet

clovaai / overhaul-distillation

sony / model_optimization

A-suozhang / awesome-quantization-and-fixed-point-training

jshilong / FisherPruning

uber-research / permute-quantize-finetune

bhheo / AB_distillation

musco-ai / musco-pytorch

bhheo / BSS_distillation

ofsoundof / group_sparsity

ofsoundof / dhp

ofsoundof / learning_filter_basis

Zhengyu-Li / Deep-Network-Compression-based-on-Student-Teacher-Network-

cambridge-mlg / arch_uncert

yukaikw / Machine-Learning

musco-ai / musco-tf

pvtien96 / CORING

malena1906 / Pruning-Weights-with-Biobjective-Optimization-Keras

yuxwind / ExactCompression

kartikgupta-at-anu / md-bnn

Yifan122 / NetworkCompress

IU-SAIGE / sparse_mle

quic / aimet-pages

MortalHappiness / ML2019SPRING

shaharpit809 / Deep-Learning-Models

maxblumental / network-compression

Hulalazz / Deep-Compression-AlexNet

Hulalazz / Embedded-Neural-Network