knowledge-distillation

There are 46 repositories under knowledge-distillation topic.

PaddlePaddle / PaddleClas
A treasure chest for visual classification and recognition powered by PaddlePaddle
image-classification knowledge-distillation autoaugment randaugment gridmask cutmix deit repvgg swin-transformer pretrained-models image-retrieval-system image-recognition product-recognition fastdeploy
Language:Python 5275
dkozlov / awesome-knowledge-distillation
Awesome Knowledge Distillation
knowledge-distillation knowledge-transfer teacher-student co-training distillation distillation-model model-distillation knowldge-distillation kd model-compression deep-learning
3322
huawei-noah / Pretrained-Language-Model
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
knowledge-distillation model-compression quantization pretrained-models large-scale-distributed
Language:Python 2963
intel / neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
low-precision pruning sparsity auto-tuning knowledge-distillation quantization quantization-aware-training post-training-quantization smoothquant large-language-models awq fp4 gptq int4 int8 mxformat sparsegpt
Language:Python 1991
EasyNLP
alibaba / EasyNLP
EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit
transformers bert nlp pretrained-models deep-learning pytorch fewshot-learning knowledge-distillation knowledge-pretraining text-image-retrieval text-to-image-synthesis machine-learning text-classification transfer-learning
Language:Python 1959
IDEA-Research / DWPose
"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)
controlnet knowledge-distillation pose-estimation stable-diffusion-webui
Language:Python 1915
jingyi0000 / VLM_survey
Collection of AWESOME vision-language models for vision tasks
clip computer-vision deep-learning knowledge-distillation multi-modal-model survey transfer-learning vision-language-model
1786
haitongli / knowledge-distillation-pytorch
A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility
pytorch knowledge-distillation deep-neural-networks cifar10 model-compression dark-knowledge computer-vision
Language:Python 1783
microsoft / Cream
This is a collection of our NAS and Vision Transformer work.
nas automl vision-transformer rpe vit-compression efficiency knowledge-distillation
Language:Python 1579
AberHu / Knowledge-Distillation-Zoo
Pytorch implementation of various Knowledge Distillation (KD) methods.
kd-methods knowledge-transfer knowledge-distillation kd teacher-student model-compression distillation
Language:Python 1524
microsoft / NeuronBlocks
NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego
question-answering deep-learning pytorch natural-language-processing text-classification artificial-intelligence dnn qna text-matching knowledge-distillation model-compression sequence-labeling
Language:Python 1441
szagoruyko / attention-transfer
Improving Convolutional Networks via Attention Transfer (ICLR 2017)
pytorch knowledge-distillation attention deep-learning
Language:Jupyter Notebook 1426
open-mmlab / mmrazor
OpenMMLab Model Compression Toolbox and Benchmark.
nas pruning knowledge-distillation spos darts autoslim detection segmentation classification pytorch quantization
Language:Python 1378
lxztju / pytorch_classification
利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码
pytorch image-classification deployment svm knn cnn label-smoothing densenet resnext resnet flask random-forest knowledge-distillation
Language:Jupyter Notebook 1313
torchdistill
yoshitomo-matsubara / torchdistill
A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
knowledge-distillation pytorch image-classification imagenet object-detection coco semantic-segmentation cifar10 cifar100 colab-notebook google-colab pascal-voc nlp natural-language-processing transformer glue amazon-sagemaker-lab
Language:Python 1286
huawei-noah / Efficient-Computing
Efficient computing methods developed by Huawei Noah's Ark Lab
knowledge-distillation model-compression binary-neural-networks pruning quantization self-supervised
Language:Jupyter Notebook 1118
MingSun-Tse / Efficient-Deep-Learning
Collection of recent methods on (deep) neural network compression and acceleration.
model-compression network-pruning knowledge-distillation deep-learning deep-neural-networks efficient-deep-learning
900
EasyTransfer
alibaba / EasyTransfer
EasyTransfer is designed to make the development of transfer learning in NLP applications easier.
bert nlp-applications knowledge-distillation transfer-learning
Language:Python 852
horseee / Awesome-Efficient-LLM
A curated list for Efficient Large Language Models
compression efficient-llm knowledge-distillation language-model llm llm-compression model-quantization pruning-algorithms
Language:Python 818
megvii-research / mdistiller
The official implementation of [CVPR2022] Decoupled Knowledge Distillation https://arxiv.org/abs/2203.08679 and [ICCV2023] DOT: A Distillation-Oriented Trainer https://openaccess.thecvf.com/content/ICCV2023/papers/Zhao_DOT_A_Distillation-Oriented_Trainer_ICCV_2023_paper.pdf
pytorch knowledge-distillation computer-vision deep-learning cifar coco cvpr2022 imagenet iccv2023
Language:Python 749
lhyfst / knowledge-distillation-papers
knowledge distillation papers
knowledge-distillation model-compression paper dark-knowledge reading-list
727
SforAiDl / KD_Lib
A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.
knowledge-distillation model-compression pruning quantization pytorch deep-learning-library machine-learning data-science benchmarking algorithm-implementations
Language:Python 572
yuanli2333 / Teacher-free-Knowledge-Distillation
Knowledge Distillation: CVPR2020 Oral, Revisiting Knowledge Distillation via Label Smoothing Regularization
knowledge-distillation label-smoothing paper-implementations pytorch teacher-free
Language:Python 568
chaitjo / efficient-gnns
Code and resources on scalable and efficient Graph Neural Networks
contrastive-learning efficient-deep-learning efficient-neural-networks gnn graph-convolutional-networks graph-neural-networks graph-representation-learning knowledge-distillation scalable-deep-learning
Language:Python 526
segmind / distill-sd
Segmind Distilled diffusion
distillation inference knowledge-distillation stable-diffusion
Language:Python 520
mammoth
aimagelab / mammoth
An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning
continual-learning dark-experience-replay deep-learning der experience-replay knowledge-distillation neurips2020 pytorch
Language:Python 472
clovaai / overhaul-distillation
Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)
iccv2019 knowledge-distillation knowledge-transfer network-compression teacher-student
Language:Python 410
ilovepose / fast-human-pose-estimation.pytorch
Official pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
human-pose-estimation deep-learning coco-keypoints-detection mpii-dataset mscoco-keypoint fast-pose-distillation knowledge-distillation
Language:Cuda 395
yanx27 / 2DPASS
2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds (ECCV 2022) :fire:
nuscenes semantic-segmentation semantickitti lidar-point-cloud deep-learning knowledge-distillation pytorch
Language:Python 386
lenscloth / RKD
Official pytorch Implementation of Relational Knowledge Distillation, CVPR 2019
deep-neural-networks computer-vision metric-learning knowledge-distillation deep-learning
Language:Python 381
Picsart-AI-Research / MI-GAN
[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices
image-inpainting knowledge-distillation object-removal mi-gan mobile-inpainting re-parameterization
Language:Python 380
yzhao062 / SUOD
(MLSys' 21) An Acceleration System for Large-scare Unsupervised Heterogeneous Outlier Detection (Anomaly Detection)
data-mining machine-learning anomaly-detection outlier-detection distributed-systems knowledge-distillation python machine-learning-algorithms machine-learning-library
Language:Python 373
1duo / awesome-ai-infrastructures
Infrastructures™ for Machine Learning Training/Inference in Production.
artificial-intelligence machine-learning deep-learning machine-learning-systems awesome-list deep-learning-framework kubernetes apache-spark apache-arrow apache-mesos pruning quantization knowledge-distillation model-compression federated-learning
368
HikariTJU / LD
Localization Distillation for Object Detection (CVPR 2022, TPAMI 2023)
detection knowledge-distillation ld object-detection pytorch deep-learning
Language:Python 343
yzd-v / FGD
Focal and Global Knowledge Distillation for Detectors (CVPR 2022)
deep-learning knowledge-distillation object-detection pytorch
Language:Python 333
yuanyuanli85 / Fast_Human_Pose_Estimation_Pytorch
Pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419
pytorch knowledge-distillation stacked-hourglass-networks pose-estimation
Language:Python 324

knowledge-distillation

PaddlePaddle / PaddleClas

dkozlov / awesome-knowledge-distillation

huawei-noah / Pretrained-Language-Model

intel / neural-compressor

alibaba / EasyNLP

IDEA-Research / DWPose

jingyi0000 / VLM_survey

haitongli / knowledge-distillation-pytorch

microsoft / Cream

AberHu / Knowledge-Distillation-Zoo

microsoft / NeuronBlocks

szagoruyko / attention-transfer

open-mmlab / mmrazor

lxztju / pytorch_classification

yoshitomo-matsubara / torchdistill

huawei-noah / Efficient-Computing

MingSun-Tse / Efficient-Deep-Learning

alibaba / EasyTransfer

horseee / Awesome-Efficient-LLM

megvii-research / mdistiller

lhyfst / knowledge-distillation-papers

SforAiDl / KD_Lib

yuanli2333 / Teacher-free-Knowledge-Distillation

chaitjo / efficient-gnns

segmind / distill-sd

aimagelab / mammoth

clovaai / overhaul-distillation

ilovepose / fast-human-pose-estimation.pytorch

yanx27 / 2DPASS

lenscloth / RKD

Picsart-AI-Research / MI-GAN

yzhao062 / SUOD

1duo / awesome-ai-infrastructures

HikariTJU / LD

yzd-v / FGD

yuanyuanli85 / Fast_Human_Pose_Estimation_Pytorch