rohithmukku / ML_Project

ML Course Project

ML_Project

ML Course Project

Report can be found here

Experiment - 3

Results for MNIST family of datasets.

InD	Train OOD	Test OOD	AUROC	AUPRC	FPR@95
MNIST	KMNIST FMNIST	FMNIST KMNIST Omniglot	0.99 0.98 0.99	0.99 0.98 0.99	0.002 0.02 0.0
MNIST	FMNIST Omniglot	FMNIST KMNIST Omniglot	0.98 0.89 0.99	0.98 0.91 0.99	0.02 0.2 0.007
MNIST	Omniglot KMNIST	FMNIST KMNIST Omniglot	0.91 0.90 0.99	0.92 0.91 0.99	0.15 0.18 0.004
FMNIST	MNIST KMNIST	MNIST KMNIST Omniglot	0.52 0.5 0.5	0.75 0.75 0.75	0.90 0.94 0.94
FMNIST	KMNIST Omniglot	MNIST KMNIST Omniglot	0.74 0.76 0.99	0.83 0.84 0.99	0.49 0.44 0.0
FMNIST	Omniglot MNIST	MNIST KMNIST Omniglot	0.73 0.52 0.99	0.82 0.75 0.99	0.51 0.91 0.0

Results for datasets similar to CIFAR-10.

InD	Train OOD	Test OOD	AUROC	AUPRC	FPR@95
CIFAR10	CIFAR100 SVHN	CIFAR100 SVHN STL10	0.51 0.86 0.53	0.74 0.89 0.74	0.92 0.23 0.89
CIFAR10	SVHN STL10	CIFAR100 SVHN STL10	0.52 0.65 0.5	0.59 0.82 0.57	0.94 0.92 0.95
CIFAR10	STL10 CIFAR100	CIFAR100 SVHN STL10	0.52 0.61 0.50	0.60 0.71 0.58	0.94 0.93 0.94
CIFAR100	CIFAR10 SVHN	CIFAR10 SVHN STL10	0.5 0.5 0.5	0.74 0.74 0.74	0.94 0.94 0.95
CIFAR100	SVHN STL10	CIFAR10 SVHN STL10	0.52 0.72 0.27	0.63 0.85 0.53	0.94 0.90 0.99
CIFAR100	STL10 CIFAR10	CIFAR10 SVHN STL10	0.51 0.59 0.34	0.59 0.69 0.49	0.94 0.93 0.97
SVHN	CIFAR10 CIFAR100	CIFAR10 CIFAR100 STL10	0.91 0.89 0.99	0.92 0.91 0.99	0.17 0.20 0.003
SVHN	CIFAR100 STL10	CIFAR10 CIFAR100 STL10	0.83 0.82 0.83	0.90 0.89 0.91	0.85 0.85 0.84
SVHN	STL10 CIFAR10	CIFAR10 CIFAR100 STL10	0.91 0.90 0.93	0.94 0.93 0.97	0.61 0.62 0.60