distillation

There are 6 repositories under distillation topic.

IntelLabs / distiller
Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller
pytorch pruning quantization pruning-structures jupyter-notebook network-compression deep-neural-networks regularization group-lasso distillation early-exit truncated-svd onnx automl-for-compression
Language:Jupyter Notebook 4309
dkozlov / awesome-knowledge-distillation
Awesome Knowledge Distillation
co-training deep-learning distillation distillation-model kd knowldge-distillation knowledge-distillation knowledge-transfer model-compression model-distillation teacher-student
3322
FLHonker / Awesome-Knowledge-Distillation
Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。
deep-learning distillation kd knowldge-distillation model-compression transfer-learning
2411
airaria / TextBrewer
A PyTorch-based knowledge distillation toolkit for natural language processing
bert pytorch nlp knowledge distillation
Language:Python 1559
PaddlePaddle / PaddleSlim
PaddleSlim is an open-source library for deep model compression and architecture search.
pruning quantization nas bert compression detection distillation ernie segmentation sparsity tensorrt transformer yolov6 yolov5 yolov7
Language:Python 1526
AberHu / Knowledge-Distillation-Zoo
Pytorch implementation of various Knowledge Distillation (KD) methods.
kd-methods knowledge-transfer knowledge-distillation kd teacher-student model-compression distillation
Language:Python 1523
GMvandeVen / continual-learning
PyTorch implementation of various methods for continual learning (XdG, EWC, SI, LwF, FROMP, DGR, BI-R, ER, A-GEM, iCaRL, Generative Classifier) in three different scenarios.
deep-learning artificial-neural-networks continual-learning lifelong-learning incremental-learning replay distillation generative-models variational-autoencoder elastic-weight-consolidation replay-through-feedback icarl gradient-episodic-memory task-incremental-learning class-incremental-learning domain-incremental-learning
Language:Jupyter Notebook 1469
ViTAE-Transformer / ViTPose
The official repo for [NeurIPS'22] "ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation" and [TPAMI'23] "ViTPose++: Vision Transformer for Generic Body Pose Estimation"
deep-learning distillation mae pose-estimation pytorch self-supervised-learning vision-transformer
Language:Python 1191
tangxyw / RecSysPapers
推荐/广告/搜索领域工业界经典以及最前沿论文集合。A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.
calibration causal-inference cold-start contrastive-learning debias distillation diverse fairness feedback-delay look-alike match multi-modal multi-scenario multi-task papers pre-rank rank re-rank recommendation-system reinforcement-learning
Language:Python 1011
Syencil / mobile-yolov5-pruning-distillation
mobilev2-yolov5s剪枝、蒸馏，支持ncnn，tensorRT部署。ultra-light but better performence！
mobile-yolov5s distillation pruning ncnn yolov5
Language:Jupyter Notebook 813
CLUEbenchmark / CLUEPretrainedModels
高质量中文预训练模型集合：最先进大模型、最快小模型、相似度专门模型
albert bert chinese corpus dataset distillation pretrained-models roberta semantic-similarity sentence-analysis sentence-classification sentence-pairs text-classification
Language:Python 788
szq0214 / MEAL-V2
MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks. In NeurIPS 2020 workshop.
compression-algorithm distillation distillation-model efficientnet imagenet imagenet-dataset meal mobilenetv3 model-architecture pre-trained pytorch resnet50
Language:Python 684
Tanuki / tanuki.py
Prompt engineering for developers
ai chat-gpt gpt-4 gpt-4-api llm pydantic python alignment tdd ai-functions distillation tanuki
Language:Python 637
segmind / distill-sd
Segmind Distilled diffusion
distillation inference knowledge-distillation stable-diffusion
Language:Python 519
thu-ml / ares
A Python library for adversarial machine learning focusing on benchmarking adversarial robustness.
adversarial-machine-learning benchmark-framework adversarial-attacks adversarial-robustness fgsm bim mi-fgsm deepfool nes spsa boundary evolutionary mmlda distillation hgd pca
Language:Python 460
anarchy-ai / LLM-VM
irresponsible innovation. Try now at https://chat.dev/
artificial-intelligence deep-learning distillation distillation-model llm llm-agent llm-inference llm-local llm-training machine-learning
Language:Python 455
Zhen-Dong / HAWQ
Quantization library for PyTorch. Support low-precision and mixed-precision quantization, with hardware implementation through TVM.
quantization tvm model-compression distillation quantized-neural-networks pytorch hardware-aware mixed-precision efficient-neural-networks 8-bit 4-bit tensorcore hessian
Language:Python 399
dotchen / LAV
(CVPR 2022) A minimalist, mapless, end-to-end self-driving stack for joint perception, prediction, planning and control.
autonomous-driving distillation carla-simulator cvpr2022 imitation-learning perception prediction planning
Language:Python 381
huggingface / optimum-intel
🤗 Optimum Intel: Accelerate inference with Intel optimization tools
diffusers distillation inference intel onnx openvino optimization pruning quantization transformers
Language:Jupyter Notebook 330
qiangsiwei / bert_distill
BERT distillation（基于BERT的蒸馏实验）
bert classification distillation nlp
Language:Python 304
gyunggyung / AGI-Papers
Papers and Book to look at when starting AGI 📚
nlg llm nlp dialogue sentence-similarity efficient distillation multimodal sentence-embeddings multiple-tasks stable-diffusion text-to-video tts all-to-all
247
leondgarse / Keras_insightface
Insightface Keras implementation
keras tensorflow2 insightface triplet arcface efficientnet subcenter-arcface distillation ghostnet tensorflow tf magface curricularface vargface
Language:Python 223
GMvandeVen / brain-inspired-replay
A brain-inspired version of generative replay for continual learning with deep neural networks (e.g., class-incremental learning on CIFAR-100; PyTorch code).
continual-learning lifelong-learning incremental-learning generative-replay brain-inspired deep-learning artificial-neural-networks split-mnist permuted-mnist split-cifar100 internal-replay replay-through-feedback replay synaptic-intelligence elastic-weight-consolidation variational-autoencoder distillation
Language:Python 217
HoyTta0 / KnowledgeDistillation
Knowledge distillation in text classification with pytorch. 知识蒸馏，中文文本分类，教师模型BERT、XLNET，学生模型biLSTM。
bert distillation knowledge-distillation model-compression pytorch
Language:Python 205
Sharpiless / Yolov5-distillation-train-inference
Yolov5 distillation training | Yolov5知识蒸馏训练，支持训练自己的数据
yolov5 distillation object-detection model-compression konwledge-distillation
Language:Python 201
Nota-NetsPresso / BK-SDM
A Compressed Stable Diffusion for Efficient Text-to-Image Generation [ICCV'23 Demo] [ICML'23 Workshop]
compression distillation huggingface lightweight pytorch stable-diffusion
Language:Python 195
snap-research / R2L
[ECCV 2022] R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis
deep-learning distillation mlp nerf neural-light-field novel-view-synthesis rendering
Language:Python 183
monologg / DistilKoBERT
Distillation of KoBERT from SKTBrain (Lightweight KoBERT)
distillation kobert bert pytorch transformers lightweight korean-nlp
Language:Python 181
szq0214 / FKD
Official code for our ECCV'22 paper "A Fast Knowledge Distillation Framework for Visual Recognition"
knowledge-distillation efficient-training efficient-algorithm pytorch self-supervised-learning deep-learning efficientnet-pytorch distillation
Language:Python 176
biosteam
BioSTEAMDevelopmentGroup / biosteam
The Biorefinery Simulation and Techno-Economic Analysis Modules; Life Cycle Assessment; Chemical Process Simulation Under Uncertainty
distillation flash biorefinery bioprocess fermentation thermodynamics pump monte-carlo heat-exchanger techno-economic-analysis chemical-engineering biochemical-process unit-operation process-simulation sensitivity-analysis reactor centrifuge life-cycle-assessment
Language:Python 170
dotchen / WorldOnRails
(ICCV 2021, Oral) RL and distillation in CARLA using a factorized world model
reinforcement-learning distillation autonomous-driving carla-simulator iccv2021
Language:Python 159
fxmeng / filter-grafting
Filter Grafting for Deep Neural Networks(CVPR 2020)
filter grafting pruning distillation multi-network pytorch
Language:Python 140
geyingli / unif
基于 Tensorflow，仿 Scikit-Learn 设计的深度学习自然语言处理框架。支持 40 余种模型类，涵盖语言模型、文本分类、NER、MRC、知识蒸馏等各个领域
nlp tensorflow language-modeling mrc classification transformer distillation gpu-training bert deep-learning
Language:Python 113
as791 / Adversarial-Example-Attack-and-Defense
This repository contains the implementation of three adversarial example attack methods FGSM, IFGSM, MI-FGSM and one Distillation as defense against all attacks using MNIST dataset.
pytorch-implementation adversarial-attacks adversarial-defense adversarial-examples mi-fgsm defense distillation attack temperature fgsm
Language:Jupyter Notebook 106
khurramjaved96 / incremental-learning
Pytorch implementation of ACCV18 paper "Revisiting Distillation and Incremental Classifier Learning."
machine-learning convolutional-neural-networks pytorch incremental-learning distillation paper-implementations
Language:Python 106
htqin / awesome-efficient-aigc
A list of papers, docs, codes about efficient AIGC. This repo is aimed to provide the info for efficient AIGC research, including language and vision, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.
aigc diffusion-models distillation efficient-deep-learning generative-model large-language-models llm model-compression model-quantization pruning
104

distillation

IntelLabs / distiller

dkozlov / awesome-knowledge-distillation

FLHonker / Awesome-Knowledge-Distillation

airaria / TextBrewer

PaddlePaddle / PaddleSlim

AberHu / Knowledge-Distillation-Zoo

GMvandeVen / continual-learning

ViTAE-Transformer / ViTPose

tangxyw / RecSysPapers

Syencil / mobile-yolov5-pruning-distillation

CLUEbenchmark / CLUEPretrainedModels

szq0214 / MEAL-V2

Tanuki / tanuki.py

segmind / distill-sd

thu-ml / ares

anarchy-ai / LLM-VM

Zhen-Dong / HAWQ

dotchen / LAV

huggingface / optimum-intel

qiangsiwei / bert_distill

gyunggyung / AGI-Papers

leondgarse / Keras_insightface

GMvandeVen / brain-inspired-replay

HoyTta0 / KnowledgeDistillation

Sharpiless / Yolov5-distillation-train-inference

Nota-NetsPresso / BK-SDM

snap-research / R2L

monologg / DistilKoBERT

szq0214 / FKD

BioSTEAMDevelopmentGroup / biosteam

dotchen / WorldOnRails

fxmeng / filter-grafting

geyingli / unif

as791 / Adversarial-Example-Attack-and-Defense

khurramjaved96 / incremental-learning

htqin / awesome-efficient-aigc