knowledge-distillation

There are 31 repositories under knowledge-distillation topic.

PaddlePaddle / PaddleClas
A treasure chest for visual classification and recognition powered by PaddlePaddle
image-classification knowledge-distillation autoaugment randaugment gridmask cutmix deit repvgg swin-transformer pretrained-models image-retrieval-system image-recognition product-recognition fastdeploy
Language:Python 4716
dkozlov / awesome-knowledge-distillation
Awesome Knowledge Distillation
knowledge-distillation knowledge-transfer teacher-student co-training distillation distillation-model model-distillation knowldge-distillation kd model-compression deep-learning
3030
huawei-noah / Pretrained-Language-Model
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
knowledge-distillation model-compression quantization pretrained-models large-scale-distributed
Language:Python 2648
haitongli / knowledge-distillation-pytorch
A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
pytorch knowledge-distillation deep-neural-networks cifar10 model-compression dark-knowledge computer-vision
Language:Python 1564
EasyNLP
alibaba / EasyNLP
EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit
transformers bert nlp pretrained-models deep-learning pytorch fewshot-learning knowledge-distillation knowledge-pretraining text-image-retrieval text-to-image-synthesis machine-learning text-classification transfer-learning
Language:Python 1494
microsoft / NeuronBlocks
NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego
question-answering deep-learning pytorch natural-language-processing text-classification artificial-intelligence dnn qna text-matching knowledge-distillation model-compression sequence-labeling
Language:Python 1428
szagoruyko / attention-transfer
Improving Convolutional Networks via Attention Transfer (ICLR 2017)
pytorch knowledge-distillation attention deep-learning
Language:Jupyter Notebook 1384
AberHu / Knowledge-Distillation-Zoo
Pytorch implementation of various Knowledge Distillation (KD) methods.
kd-methods knowledge-transfer knowledge-distillation kd teacher-student model-compression distillation
Language:Python 1226
lxztju / pytorch_classification
利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码
pytorch image-classification deployment svm knn cnn label-smoothing densenet resnext resnet flask random-forest knowledge-distillation
Language:Jupyter Notebook 1093
intel / neural-compressor
Intel® Neural Compressor (formerly known as Intel® Low Precision Optimization Tool), targeting to provide unified APIs for network compression technologies, such as low precision quantization, sparsity, pruning, knowledge distillation, across different deep learning frameworks to pursue optimal inference performance.
auto-tuning deep-learning knowledge-distillation low-precision post-training-quantization pruning quantization quantization-aware-training sparsity
Language:Python 993
open-mmlab / mmrazor
OpenMMLab Model Compression Toolbox and Benchmark.
autoslim classification darts detection knowledge-distillation nas pruning pytorch segmentation spos
Language:Python 983
microsoft / Cream
This is a collection of our NAS and Vision Transformer work.
nas automl vision-transformer rpe vit-compression efficiency knowledge-distillation
Language:Python 883
yoshitomo-matsubara / torchdistill
A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆20 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
knowledge-distillation pytorch image-classification imagenet object-detection coco semantic-segmentation cifar10 cifar100 colab-notebook google-colab pascal-voc nlp natural-language-processing transformer glue
Language:Python 868
EasyTransfer
alibaba / EasyTransfer
EasyTransfer is designed to make the development of transfer learning in NLP applications easier.
bert knowledge-distillation nlp-applications transfer-learning
Language:Python 827
MingSun-Tse / Efficient-Deep-Learning
Collection of recent methods on (deep) neural network compression and acceleration.
model-compression network-pruning knowledge-distillation deep-learning deep-neural-networks efficient-deep-learning
776
lhyfst / knowledge-distillation-papers
knowledge distillation papers
knowledge-distillation model-compression paper dark-knowledge reading-list
647
huawei-noah / Efficient-Computing
Efficient-Computing
knowledge-distillation model-compression
Language:Python 556
yuanli2333 / Teacher-free-Knowledge-Distillation
Knowledge Distillation: CVPR2020 Oral, Revisiting Knowledge Distillation via Label Smoothing Regularization
knowledge-distillation pytorch teacher-free paper-implementations label-smoothing
Language:Python 514
chaitjo / efficient-gnns
Code and resources on scalable and efficient Graph Neural Networks
contrastive-learning efficient-deep-learning efficient-neural-networks gnn graph-convolutional-networks graph-neural-networks graph-representation-learning knowledge-distillation scalable-deep-learning
Language:Python 507
megvii-research / mdistiller
The official implementation of [CVPR2022] Decoupled Knowledge Distillation https://arxiv.org/abs/2203.08679
cifar coco computer-vision cvpr2022 deep-learning imagenet knowledge-distillation pytorch
Language:Python 505
SforAiDl / KD_Lib
A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.
knowledge-distillation model-compression pruning quantization pytorch deep-learning-library machine-learning data-science benchmarking algorithm-implementations
Language:Python 487
clovaai / overhaul-distillation
Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)
knowledge-distillation knowledge-transfer teacher-student network-compression iccv2019
Language:Python 391
ilovepose / fast-human-pose-estimation.pytorch
Official pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
human-pose-estimation deep-learning coco-keypoints-detection mpii-dataset mscoco-keypoint fast-pose-distillation knowledge-distillation
Language:Cuda 381
yzhao062 / SUOD
(MLSys' 21) An Acceleration System for Large-scare Unsupervised Heterogeneous Outlier Detection (Anomaly Detection)
data-mining machine-learning anomaly-detection outlier-detection distributed-systems knowledge-distillation python machine-learning-algorithms machine-learning-library
Language:Python 345
lenscloth / RKD
Official pytorch Implementation of Relational Knowledge Distillation, CVPR 2019
computer-vision deep-learning deep-neural-networks knowledge-distillation metric-learning
Language:Python 338
yuanyuanli85 / Fast_Human_Pose_Estimation_Pytorch
Pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
knowledge-distillation pose-estimation pytorch stacked-hourglass-networks
Language:Python 327
mammoth
aimagelab / mammoth
An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning
pytorch continual-learning neurips2020 knowledge-distillation deep-learning der experience-replay dark-experience-replay
Language:Python 317
1duo / awesome-ai-infrastructures
Infrastructures™ for Machine Learning Training/Inference in Production.
artificial-intelligence machine-learning deep-learning machine-learning-systems awesome-list deep-learning-framework kubernetes apache-spark apache-arrow apache-mesos pruning quantization knowledge-distillation model-compression federated-learning
311
HikariTJU / LD
Localization Distillation for Dense Object Detection (CVPR 2022)
detection knowledge-distillation ld object-detection pytorch deep-learning
Language:Python 296
yzd-v / FGD
Focal and Global Knowledge Distillation for Detectors (CVPR 2022)
object-detection knowledge-distillation deep-learning pytorch
Language:Python 276
yanx27 / 2DPASS
2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds (ECCV 2022) :fire:
nuscenes semantic-segmentation semantickitti lidar-point-cloud deep-learning knowledge-distillation pytorch
Language:Python 271
KD_methods_with_TF
sseung0703 / KD_methods_with_TF
Knowledge distillation methods implemented with Tensorflow (now there are 11 (+1) methods, and will be added more.)
knowledge-distillation tensorflow-examples tensorflow
Language:Python 259
imirzadeh / Teacher-Assistant-Knowledge-Distillation
Using Teacher Assistants to Improve Knowledge Distillation: https://arxiv.org/pdf/1902.03393.pdf
deep-learning knowledge-distillation neural-network-compression
Language:Python 230
sebastian-hofstaetter / matchmaker
Training & evaluation library for text-based neural re-ranking and dense retrieval models built with PyTorch
neural-ir transformer explainability efficiency knowledge-distillation dpr pytorch neural-ranking
Language:Python 210
karanchahal / distiller
A large scale study of Knowledge Distillation.
feature-distillation knowledge-distillation pytorch-lightning
Language:Python 207
nathanhubens / fasterai
FasterAI: Prune and Distill your models with FastAI and PyTorch
pruning compression fastai pytorch knowledge-distillation
Language:Jupyter Notebook 200

knowledge-distillation

PaddlePaddle / PaddleClas

dkozlov / awesome-knowledge-distillation

huawei-noah / Pretrained-Language-Model

haitongli / knowledge-distillation-pytorch

alibaba / EasyNLP

microsoft / NeuronBlocks

szagoruyko / attention-transfer

AberHu / Knowledge-Distillation-Zoo

lxztju / pytorch_classification

intel / neural-compressor

open-mmlab / mmrazor

microsoft / Cream

yoshitomo-matsubara / torchdistill

alibaba / EasyTransfer

MingSun-Tse / Efficient-Deep-Learning

lhyfst / knowledge-distillation-papers

huawei-noah / Efficient-Computing

yuanli2333 / Teacher-free-Knowledge-Distillation

chaitjo / efficient-gnns

megvii-research / mdistiller

SforAiDl / KD_Lib

clovaai / overhaul-distillation

ilovepose / fast-human-pose-estimation.pytorch

yzhao062 / SUOD

lenscloth / RKD

yuanyuanli85 / Fast_Human_Pose_Estimation_Pytorch

aimagelab / mammoth

1duo / awesome-ai-infrastructures

HikariTJU / LD

yzd-v / FGD

yanx27 / 2DPASS

sseung0703 / KD_methods_with_TF

imirzadeh / Teacher-Assistant-Knowledge-Distillation

sebastian-hofstaetter / matchmaker

karanchahal / distiller

nathanhubens / fasterai