Kim-Minseon / RoCL

Code for the paper "Adversarial Self-supervised Contrastive Learning" (NeurIPS 2020)

RoCL-Adversarial self-supervised contrastive learning

This repository is the official PyTorch implementation of "Adversarial self supervised contrastive learning" by Minseon Kim, Jihoon Tack and Sung Ju Hwang.

Requirements

Currently, requires following packages

• python 3.6+
• torch 1.1+
• torchvision 0.3+
• CUDA 10.1+
• torchlars == 0.1.2
• pytorch-gradual-warmup-lr packages
• diffdist == 0.1

Training

To train the model(s) in the paper, run this command:

mkdir Data folder inside the RoCL

mkdir ./Data

python -m torch.distributed.launch --nproc_per_node=2 rocl_train.py --ngpu 2 --batch-size=256 --model='ResNet18' --k=7 --loss_type='sim' --advtrain_type='Rep' --attack_type='linf' --name=<name-of-the-file> --regularize_to='other' --attack_to='other' --train_type='contrastive' --dataset='cifar-10'

Evaluation

To evaluate my model linear evaluation and robustness, run:

./total_process.sh test <checkpoint-load> <name> <model type='ResNet18' or 'ResNet50'> <learning rate=0.1> <dataset='cifar-10' or 'cifar-100'>

Results

Our model achieves the following performance on :

Classification and robustness on CIFAR 10

Model name	Accuracy	robustness
RoCL ResNet18	83.71 %	40.27%

Citation

@inproceedings{kim2020adversarial,
  title={Adversarial Self-Supervised Contrastive Learning},
  author={Minseon Kim and Jihoon Tack and Sung Ju Hwang},
  booktitle = {Advances in Neural Information Processing Systems},
  year={2020}
}