distillation

There are 7 repositories under distillation topic.

IntelLabs / distiller
Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller
automl-for-compression deep-neural-networks distillation early-exit group-lasso jupyter-notebook network-compression onnx pruning pruning-structures pytorch quantization regularization truncated-svd
Language:Jupyter Notebook 4328
dkozlov / awesome-knowledge-distillation
Awesome Knowledge Distillation
co-training deep-learning distillation distillation-model kd knowldge-distillation knowledge-distillation knowledge-transfer model-compression model-distillation teacher-student
3375
FLHonker / Awesome-Knowledge-Distillation
Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。
deep-learning distillation kd knowldge-distillation model-compression transfer-learning
2440
airaria / TextBrewer
A PyTorch-based knowledge distillation toolkit for natural language processing
bert distillation knowledge nlp pytorch
Language:Python 1571
AberHu / Knowledge-Distillation-Zoo
Pytorch implementation of various Knowledge Distillation (KD) methods.
kd-methods knowledge-transfer knowledge-distillation kd teacher-student model-compression distillation
Language:Python 1550
PaddlePaddle / PaddleSlim
PaddleSlim is an open-source library for deep model compression and architecture search.
pruning quantization nas bert compression detection distillation ernie segmentation sparsity tensorrt transformer yolov6 yolov5 yolov7
Language:Python 1545
GMvandeVen / continual-learning
PyTorch implementation of various methods for continual learning (XdG, EWC, SI, LwF, FROMP, DGR, BI-R, ER, A-GEM, iCaRL, Generative Classifier) in three different scenarios.
artificial-neural-networks class-incremental-learning continual-learning deep-learning distillation domain-incremental-learning elastic-weight-consolidation generative-models gradient-episodic-memory icarl incremental-learning lifelong-learning replay replay-through-feedback task-incremental-learning variational-autoencoder
Language:Jupyter Notebook 1506
ViTAE-Transformer / ViTPose
The official repo for [NeurIPS'22] "ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation" and [TPAMI'23] "ViTPose++: Vision Transformer for Generic Body Pose Estimation"
deep-learning distillation mae pose-estimation pytorch self-supervised-learning vision-transformer
Language:Python 1275
tangxyw / RecSysPapers
推荐/广告/搜索领域工业界经典以及最前沿论文集合。A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.
calibration causal-inference cold-start contrastive-learning debias distillation diverse fairness match papers rank re-rank recommendation-system feedback-delay pre-rank look-alike multi-scenario multi-task reinforcement-learning multi-modal
Language:Python 1155
Syencil / mobile-yolov5-pruning-distillation
mobilev2-yolov5s剪枝、蒸馏，支持ncnn，tensorRT部署。ultra-light but better performence！
mobile-yolov5s distillation pruning ncnn yolov5
Language:Jupyter Notebook 817
CLUEbenchmark / CLUEPretrainedModels
高质量中文预训练模型集合：最先进大模型、最快小模型、相似度专门模型
pretrained-models chinese bert roberta albert sentence-classification sentence-pairs semantic-similarity sentence-analysis text-classification distillation corpus dataset
Language:Python 795
szq0214 / MEAL-V2
MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks. In NeurIPS 2020 workshop.
distillation imagenet compression-algorithm mobilenetv3 efficientnet pre-trained resnet50 imagenet-dataset meal model-architecture pytorch distillation-model
Language:Python 688
Tanuki / tanuki.py
Prompt engineering for developers
ai chat-gpt gpt-4 gpt-4-api llm pydantic python alignment tdd ai-functions distillation tanuki
Language:Python 654
segmind / distill-sd
Segmind Distilled diffusion
distillation inference knowledge-distillation stable-diffusion
Language:Python 533
thu-ml / ares
A Python library for adversarial machine learning focusing on benchmarking adversarial robustness.
adversarial-machine-learning benchmark-framework adversarial-attacks adversarial-robustness fgsm bim mi-fgsm deepfool nes spsa boundary evolutionary mmlda distillation hgd pca
Language:Python 471
anarchy-ai / LLM-VM
irresponsible innovation. Try now at https://chat.dev/
artificial-intelligence deep-learning distillation distillation-model llm llm-agent llm-inference llm-local llm-training machine-learning
Language:Python 469
Zhen-Dong / HAWQ
Quantization library for PyTorch. Support low-precision and mixed-precision quantization, with hardware implementation through TVM.
quantization tvm model-compression distillation quantized-neural-networks pytorch hardware-aware mixed-precision efficient-neural-networks 8-bit 4-bit tensorcore hessian
Language:Python 406
gojasper / flash-diffusion
Official implementation of ⚡ Flash Diffusion ⚡: Accelerating Any Conditional Diffusion Model for Few Steps Image Generation
diffusion-models distillation dit inpainting sdxl super-resolution text-to-image
Language:Python 397
dotchen / LAV
(CVPR 2022) A minimalist, mapless, end-to-end self-driving stack for joint perception, prediction, planning and control.
autonomous-driving distillation carla-simulator cvpr2022 imitation-learning perception prediction planning
Language:Python 390
huggingface / optimum-intel
🤗 Optimum Intel: Accelerate inference with Intel optimization tools
diffusers distillation inference intel onnx openvino optimization pruning quantization transformers
Language:Jupyter Notebook 361
qiangsiwei / bert_distill
BERT distillation（基于BERT的蒸馏实验）
bert classification distillation nlp
Language:Python 307
gyunggyung / AGI-Papers
Papers and Book to look at when starting AGI 📚
all-to-all dialogue distillation efficient llm multimodal multiple-tasks nlg nlp sentence-embeddings sentence-similarity stable-diffusion text-to-video tts
257
leondgarse / Keras_insightface
Insightface Keras implementation
keras tensorflow2 insightface triplet arcface efficientnet subcenter-arcface distillation ghostnet tensorflow tf magface curricularface vargface
Language:Python 230
GMvandeVen / brain-inspired-replay
A brain-inspired version of generative replay for continual learning with deep neural networks (e.g., class-incremental learning on CIFAR-100; PyTorch code).
continual-learning lifelong-learning incremental-learning generative-replay brain-inspired deep-learning artificial-neural-networks split-mnist permuted-mnist split-cifar100 internal-replay replay-through-feedback replay synaptic-intelligence elastic-weight-consolidation variational-autoencoder distillation
Language:Python 223
Nota-NetsPresso / BK-SDM
A Compressed Stable Diffusion for Efficient Text-to-Image Generation [ECCV'24]
compression stable-diffusion huggingface pytorch distillation lightweight
Language:Python 216
HoyTta0 / KnowledgeDistillation
Knowledge distillation in text classification with pytorch. 知识蒸馏，中文文本分类，教师模型BERT、XLNET，学生模型biLSTM。
bert distillation knowledge-distillation model-compression pytorch
Language:Python 209
Sharpiless / Yolov5-distillation-train-inference
Yolov5 distillation training | Yolov5知识蒸馏训练，支持训练自己的数据
distillation konwledge-distillation model-compression object-detection yolov5
Language:Python 200
monologg / DistilKoBERT
Distillation of KoBERT from SKTBrain (Lightweight KoBERT)
bert distillation kobert korean-nlp lightweight pytorch transformers
Language:Python 184
snap-research / R2L
[ECCV 2022] R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis
nerf deep-learning distillation neural-light-field mlp novel-view-synthesis rendering
Language:Python 184
szq0214 / FKD
Official code for our ECCV'22 paper "A Fast Knowledge Distillation Framework for Visual Recognition"
knowledge-distillation efficient-training efficient-algorithm pytorch self-supervised-learning deep-learning efficientnet-pytorch distillation
Language:Python 176
biosteam
BioSTEAMDevelopmentGroup / biosteam
The Biorefinery Simulation and Techno-Economic Analysis Modules; Life Cycle Assessment; Chemical Process Simulation Under Uncertainty
distillation flash biorefinery bioprocess fermentation thermodynamics pump monte-carlo heat-exchanger techno-economic-analysis chemical-engineering biochemical-process unit-operation process-simulation sensitivity-analysis reactor centrifuge life-cycle-assessment
Language:Python 172
dotchen / WorldOnRails
(ICCV 2021, Oral) RL and distillation in CARLA using a factorized world model
reinforcement-learning distillation autonomous-driving carla-simulator iccv2021
Language:Python 163
fxmeng / filter-grafting
Filter Grafting for Deep Neural Networks(CVPR 2020)
filter grafting pruning distillation multi-network pytorch
Language:Python 140
htqin / awesome-efficient-aigc
A list of papers, docs, codes about efficient AIGC. This repo is aimed to provide the info for efficient AIGC research, including language and vision, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.
aigc diffusion-models distillation efficient-deep-learning generative-model large-language-models llm model-compression model-quantization pruning
120
geyingli / unif
基于 Tensorflow，仿 Scikit-Learn 设计的深度学习自然语言处理框架。支持 40 余种模型类，涵盖语言模型、文本分类、NER、MRC、知识蒸馏等各个领域
nlp tensorflow language-modeling mrc classification transformer distillation gpu-training bert deep-learning
Language:Python 113
as791 / Adversarial-Example-Attack-and-Defense
This repository contains the implementation of three adversarial example attack methods FGSM, IFGSM, MI-FGSM and one Distillation as defense against all attacks using MNIST dataset.
adversarial-attacks adversarial-defense adversarial-examples attack defense distillation fgsm mi-fgsm pytorch-implementation temperature
Language:Jupyter Notebook 111

distillation

IntelLabs / distiller

dkozlov / awesome-knowledge-distillation

FLHonker / Awesome-Knowledge-Distillation

airaria / TextBrewer

AberHu / Knowledge-Distillation-Zoo

PaddlePaddle / PaddleSlim

GMvandeVen / continual-learning

ViTAE-Transformer / ViTPose

tangxyw / RecSysPapers

Syencil / mobile-yolov5-pruning-distillation

CLUEbenchmark / CLUEPretrainedModels

szq0214 / MEAL-V2

Tanuki / tanuki.py

segmind / distill-sd

thu-ml / ares

anarchy-ai / LLM-VM

Zhen-Dong / HAWQ

gojasper / flash-diffusion

dotchen / LAV

huggingface / optimum-intel

qiangsiwei / bert_distill

gyunggyung / AGI-Papers

leondgarse / Keras_insightface

GMvandeVen / brain-inspired-replay

Nota-NetsPresso / BK-SDM

HoyTta0 / KnowledgeDistillation

Sharpiless / Yolov5-distillation-train-inference

monologg / DistilKoBERT

snap-research / R2L

szq0214 / FKD

BioSTEAMDevelopmentGroup / biosteam

dotchen / WorldOnRails

fxmeng / filter-grafting

htqin / awesome-efficient-aigc

geyingli / unif

as791 / Adversarial-Example-Attack-and-Defense