out-of-distribution-detection
This repo is a pytorch implementation of Out-of-Distribution Detection Using an Ensembleof Self Supervised Leave-out Classifiers realised within the course DD2412 Deep Learning, Advanced at KTH university.
Data
- CIFAR10
- Tiny-ImageNet (crop) (provided by fb-research)
- Tiny-ImageNet (resize) (provided by fb-research)
- LSUN (crop) (provided by fb-research)
- LSUN (resize) (provided by fb-research)
- iSUN (provided by fb-research)
Uniform Noise (UNFM) and Gaussian Noise (GSSN) can be generated thanks to the script noise_ood_datasets_generator.py
Project structure
The project is structured as following:
├── data # data should be upload here
├── datasets
| └── cifar10.py # a custom dataset design to handle margin_loss with cifar10
| └── ood.py # basic custom dataset for ood data
├── distributions # has to be created, distribution plots (id vs ood) will be saved here
├── models
| └── dense_net.py # densenet nn
| └── toy_net.py # a basic cnn taken from the pytorch cifar10 tutorial
| └── wide_res_net.py # wideresnet nn
├── distributions # has to be created, models weights will be saved here
├── class_to_id_lists.py
├── classifier.py # definition of the Classifier class
├── loss.py # entropy based margin loss
├── metrics.py # detection error, fpr95, auroc, aupr in, aupr out
├── noise_ood_datasets_generator.py # has to be run to generate uniform noise and gaussian noise datasets
├── ood_validation.py # ood scores computation
├── train.py # pipelines for training
├── test.y # pipelines for testing
Launching
Train: execute train.py, parameters have to be chosen into the file.
Validation is executed on iSUN with T=100, epsilon=0.002.
Test: execute test.py, net_architecture and training_name have to be consistent.
The results have been obtained by executing this file as is.
The best classifiers set weights are available here.
Project made by Simon Lembeye, Emile Lucas and Muhammed Memedi.