self-distillation

There are 1 repository under self-distillation topic.

Tebmer / Awesome-Knowledge-Distillation-of-LLMs
This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.
data-augmentation instruction-following kd knowledge-distillation large-language-model llm self-training survey compression data-synthesis feedback multi-modal self-distillation alignment supervised-finetuning
295
luanyunteng / pytorch-be-your-own-teacher
A pytorch implementation of paper 'Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation', https://arxiv.org/abs/1905.08094
knowledge-distillation self-distillation
Language:Python 144
sail-sg / sdft
[ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".
language-model self-distillation supervised-finetuning
Language:Shell 42
youngkyunJang / Deep-Hash-Distillation
Deep Hash Distillation for Image Retrieval - ECCV 2022
deep-learning hashing image-retrieval pytorch representation-learning self-distillation vision-transformer
Language:Python 40
Kennethborup / self_distillation
Self-Distillation with weighted ground-truth targets; ResNet and Kernel Ridge Regression
kernel-ridge-regression distillation knowledge-distillation self-distillation resnet ridge-regression self-training
Language:Jupyter Notebook 17
youngerous / ddgsd-pytorch
(Unofficial) Data-Distortion Guided Self-Distillation for Deep Neural Networks (AAAI 2019)
ddgsd knowledge-distillation pytorch self-distillation
Language:Python 14
mrpositron / distillation
Self-Distillation and Knowledge Distillation Experiments with PyTorch.
deep-learning distillation knowledge-distillation pytorch self-distillation
Language:Python 9
YeongHyeon / DINO_MNIST-PyTorch
Pytorch implementation of "Emerging Properties in Self-Supervised Vision Transformers" (a.k.a. DINO)
knowledge-distillation mnist mnist-dataset pytorch self-distillation self-supervised-learning torch toy-example
Language:Python 9
dongkyuk / DLB-Pytorch
A minimalist unofficial implementation of "Self-Distillation from the Last Mini-Batch for Consistency Regularization"
pytorch knowlege-distillation self-distillation cifar10 computer-vision deep-learning
Language:Python 8
sooperset / boss
Bayesian Optimization Meets Self-Distillation, ICCV 2023
computer-vision deep-learning hyperparameter-optimization iccv2023 self-distillation
Language:Python 8
marcomoldovan / multimodal-self-distillation
A generalized self-supervised training paradigm for unimodal and multimodal alignment and fusion.
multimodal-deep-learning pytorch self-distillation self-supervised-learning multimodal-alignment multimodal-fusion multimodal-retrieval
Language:Python 5
Kennethborup / gaussian_process_self_distillation
Official implementation of Self-Distillation for Gaussian Processes
distillation gaussian-process-classifiers gaussian-process-regression gaussian-processes knowledge-distillation self-distillation
Language:Python 3
d-f / dino-vit-pcam
Self supervised learning through self distillation with no labels (DINO) with Vision Transformers on the PCAM dataset.
deep-learning image-classification pcam-dataset pytorch self-distillation self-supervised-learning vision-transformer
Language:Python 1
kisonho / magnet
Modality-Agnostic Learning for Medical Image Segmentation Using Multi-modality Self-distillation
medical-imaging segmentation self-distillation modality-agnostic
Language:Python 0

self-distillation

Tebmer / Awesome-Knowledge-Distillation-of-LLMs

luanyunteng / pytorch-be-your-own-teacher

sail-sg / sdft

youngkyunJang / Deep-Hash-Distillation

Kennethborup / self_distillation

youngerous / ddgsd-pytorch

mrpositron / distillation

YeongHyeon / DINO_MNIST-PyTorch

dongkyuk / DLB-Pytorch

sooperset / boss

marcomoldovan / multimodal-self-distillation

Kennethborup / gaussian_process_self_distillation

d-f / dino-vit-pcam

kisonho / magnet