samunaai / gram-ood

Code for the best paper at ISIC Skin Image Analysis Workshop paper at CVPR 2020.

On Out-of-Distribution Detection Algorithms with Deep Neural Skin CancerClassifiers

Code for the best paper of the ISIC Skin Image Analysis Workshop at CVPR 2020.

Paper Download: link

Workshop Talk: link

For the Mahalanobis, ODIN, and Baseline original codes, please, refer to the following repositories:

• ODIN: Out-of-Distribution Detector for Neural Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks

How to run

• Install the dependencies: pip install -r requirements.txt
• Download data and checkpoints running the setup.sh script.
• Run the network notebook (ex: sk_mobilenet.ipynb)
• The Mahalanobis, ODIN, and Baseline adaptations are available on mahalanobis_and_odin folder.

If you don't want to use the setup.shscript, you can download data from here and the checkpoints from here.

ISIC unknown detection is available on isic_submit folder.

Results

Unbiased performance in terms of TNR @ TPR 95%

Running time for Gram-OOD and Gram-OOD*

ISIC 2019 unknown detection - Resnet-50

Team on live leaderboard: Gram-OOD - UNK detection using Gram-OOD* - Resnet

The remaining results are available on this folder