HobbitLong / RepDistiller

[ICLR 2020] Contrastive Representation Distillation (CRD), and benchmark of recent knowledge distillation methods

HobbitLong/RepDistiller Issues

Has anyone implemented Wasserstein Contrastive Representation Distillation
Updated 13 days ago1
How do you choose the optimal hyper-parameters?
Updated 2 months ago2
about using the resnet models for cifar10
Updated 2 months ago1
Ensemble Task Implementation
Updated 2 months ago2
the result is different in resnet56
Updated 8 months ago1
Failed to download the teacher models
Updated 9 months ago2
No dev set split
Closed 9 months ago
ERROR :run ./fetch_pretrained_teachers.sh
Updated 9 months ago1
How to use myself datasets?
Updated a year ago1
Is Ensemble distillation also included?
Closed 2 years ago
Hyperparameter Settings for KD on Imagenet
Updated 2 years ago
Why using log_softmax instead of softmax?
Updated 2 years ago1
Question about normalization constant Z_v1 and Z_v2 in the ContrastMemory
Updated 2 years ago
crd used in image enhancement task like Denoise\SR\Deblur.
Updated 2 years ago
resnet structure seems to be a bit wrong
Updated 2 years ago3
Problem of the order of the normalization in Similarity-Preserving loss.
Updated 3 years ago
Cross modal KD implementation release?
Updated 3 years ago1
Question on memory consumption for CRD loss when the dataset is very large
Updated 3 years ago3
AttributeError: 'CIFAR100InstanceSample' object has no attribute 'train_data'
Updated 3 years ago3
Training scheme for linear probe on STL10 and TinyImagenet
Updated 3 years ago
Error while running the code
Closed 3 years ago
Why "opt.nce_k" in dataset cifar100 is 16384? How can I get this ?
Updated 3 years ago2
Hyper-parameters for reproducing the results on ImageNet
Updated 3 years ago9
what is the difference between the position of putting "with torch.no_grad()"
Updated 3 years ago1
how to train my model?
Updated 4 years ago
About deep mutual learning setting
Updated 4 years ago
About the CE loss
Updated 4 years ago
ImageNet results
Updated 4 years ago
How to train teacher model
Updated 4 years ago
teacher model is too big to run with batch_size 64
Updated 4 years ago
Question about pretrained teacher model
Closed 4 years ago
hyperparameters for other methods
Updated 4 years ago1
the introduction of ContrastMemory
Updated 4 years ago1
Multiple GPU training
Closed 4 years ago
Reported results based on early stopping?
Closed 4 years ago3
KD method in both configurations seems to be doing better than all other methods except the one from your paper
Closed 4 years ago1
questions about ContrastMemory
Updated 4 years ago3
AttributeError: 'CIFAR100Instance' object has no attribute 'train_data'
Closed 4 years ago
How can I use CRD_loss to face landmark detetct for model compression? There is no "opt.nce_k: number of negatives paired with each positive".
Updated 4 years ago
The calculation of correlation matrix
Updated 4 years ago
code for ensemble distillation
Closed 4 years ago1
Does the crd can be applied to cross domain distillation
Closed 4 years ago2
The sampler is not consistent with the original implementation of CCKD
Closed 4 years ago1
the implementation of cckd is not consistent with the paper
Closed 4 years ago2
Form of the h function for infinite dataset
Closed 4 years ago2
Memory issue about the NST LOSS
Closed 4 years ago2
Questions about ContrastMemory
Closed 4 years ago5
Results on ImageNet
Closed 5 years ago1
In the 2 result tables, WRN-40-2, as the teacher, after distilling the students, the students get higher performance（CRD+KD）, why?
Closed 5 years ago1
Regression task
Closed 5 years ago1