This repository contains the code for our TMLR paper
Supervised Knowledge May Hurt Novel Class Discovery Performance
Ziyun Li, Jona Otholt, Ben Dai, Di Hu, Christoph Meinel, Haojin Yang
Purely self supervised learning may outperform supervised learning in NCD
To make our proposed benchmark more accessible to other researchers, we also provide it as a standalone package at https://github.com/J-L-O/NCDSimilarityBench.
We provide a conda environment file for installing the dependencies. To install the dependencies, run the following command in the root directory of this repository:
conda env create -f environment.yamlAlternatively, we also provide a Dockerfile for installing the dependencies. The Dockerfile can be built with the following command:
docker build -t skncd .The repository contains four scripts for running experiments and evaluating the results:
main_pretrain.pyis used for supervised pretraining.main_pretrain_swav.pyis used for pretraining with SwAV [2].main_discover.pyis used for discovering novel classes.main_evaluate.pyis used for evaluating both the pretraining and discovery results, e.g. by computing our proposed transfer flow.
Usage examples can be found in the scripts directory.
To reproduce the results from the paper, we provide scripts in the scripts directory.
To run them, first edit the scripts/config.sh file to point to the correct directories.
Then, run the following command:
source scripts/config.shThis will set the environment variables for the scripts. The scripts are named according to the following convention:
./scripts/<phase>_<dataset>_<pretraining>.shwhere <phase> is either pretrain or discover.
<dataset> can be either cifar50-50 for standard CIFAR100 with 50 labeled and 50 unlabeled
classes, cifar40-10 for our CIFAR100-based benchmark,
or imagenet for ImageNet for our ImageNet-based benchmark with varying semantic similarity.
<pretraining> can either be supervised for supervised pretraining, or swav for pretraining
with SwAV.
To run the experiments, always run the pretraining script first, followed by the discovery script.
Those scripts that use our benchmark datasets require the split to be specified as an argument.
In case of supervised pretraining, just the labeled split is necessary as training is only
performed on the labeled classes.
In case of SwAV pretraining, both the labeled and unlabeled split are necessary.
Possible values for the labeled split are l1, l2, and l3, where l3 corresponds to the
mixed split denoted l1.5 in the paper.
Possible values for the unlabeled split are u1, and u2.
For example, to run SwAV pretraining on our ImageNet-based benchmark for the labeled set L1 and the
unlabeled set U2, run the following command:
./scripts/pretrain_imagenet_swav.sh l1 u1For the discovery phase, we need additionally need to specify the weight alpha of the labels for the labeled targets. Alpha can be any value between 0 and 1, where 1 corresponds to standard UNO [1] and 0 to completely unsupervised learning. To continue the previous example, run the following command:
./scripts/discover_imagenet_swav.sh l1 u2 1.0to run the discovery phase using the SwAV pretrained model from the previous example and alpha=1.0. For convenience, we provide the pretrained supervised / SwAV models in this Google Drive folder.
This repository is based on the repository for UNO [1], which is licensed under the MIT license. The original license can be found in the LICENSE_UNO file.
[1] Fini, E., Sangineto, E., Lathuilière, S., Zhong, Z., Nabi, M., Ricci, E.: A unified objective for novel class discovery. In: ICCV, pp. 9284–9292 (2021)
[2] Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. Advances in Neural Information Processing Systems 33, 9912–9924 (2020)