Awesome Failure Detection / Reliable Prediction

Failure Detection is a machine learning problem, which aims to detect out-of-distribution (OOD) and misclassified samples based on reliable confidence estimation. This topic is important for risk-sensitive applications (e.g., autonomous driving, clinical decision making), and is gathering much attention in the research community.

Here, we provide a list of papers that studies OOD detection and misclassification detection (MisD).

Misclassification Detection / Selective Classification / Failure Prediction

Unified Out-Of-Distribution Detection: A Model-Specific Perspective (ICCV 2023) [paper]
OpenMix: Exploring Outlier Samples for Misclassification Detection (CVPR 2023) [paper] [code]
Towards More Reliable Confidence Estimation (TPAMI 2023) [paper]
Failure Detection for Motion Prediction of Autonomous Driving: An Uncertainty Perspective (ICRA 2023) [paper]
A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification (ICLR 2023) [paper] [code]
What can we learn from the selective prediction and uncertainty estimation performance of 523 imagenet classifiers (ICLR 2023) [paper] [code]
Towards Better Selective Classification (ICLR 2023) [paper] [code]
AUC-based Selective Classification (AISTATS 2023) [paper]
Failure Detection in Medical Image Classification: A Reality Check and Benchmarking Testbed (TMLR 2022) [paper] [code]
Rethinking Confidence Calibration for Failure Prediction (ECCV 2022) [paper] [code]
Improving the Reliability for Confidence Estimation (ECCV 2022) [paper]
Trust, but Verify: Using Self-Supervised Probing to Improve Trustworthiness (ECCV 2022) [paper]
Confidence estimation via auxiliary models (TPAMI 2021) [paper]
Learning to predict trustworthiness with steep slope loss (NeurIPS 2021) [paper] [code]
Confidence-Aware Learning for Deep Neural Networks (ICML 2020) [paper] [code]
Self-Adaptive Training: beyond Empirical Risk Minimization (NeurIPS 2020) [paper] [code]
Selectivenet: A deep neural network with an integrated reject option (ICML 2019) [paper]
Addressing Failure Prediction by Learning Model Confidence (NeurIPS 2019) [paper] [code]
Deep Gamblers: Learning to Abstain with Portfolio Theory (NeurIPS 2019) [paper] [code]
To Trust Or Not To Trust A Classifier (NeurIPS 2018) [paper] [code]
Selective classification for deep neural networks (NeurIPS 2017) [paper]
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks (ICLR 2017) [paper]
An Optimum Character Recognition System Using Decision Functions (IRE Transactions on Electronic Computers 1957) [paper]

OOD Detection

2023

Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection (Arxiv 2023) [paper]
WDiscOOD: Out-of-Distribution Detection via Whitened Linear Discriminant Analysis (ICCV 2023) [paper]
DIFFGUARD: Semantic Mismatch-Guided Out-of-Distribution Detection Using Pre-Trained Diffusion Models (ICCV 2023) [paper] [code]
Understanding the Feature Norm for Out-of-Distribution Detection (ICCV 2023) [paper]
Nearest Neighbor Guidance for Out-of-Distribution Detection (ICCV 2023) [paper]
Residual Pattern Learning for Pixel-Wise Out-of-Distribution Detection in Semantic Segmentation (ICCV 2023) [paper] [code]
Unified Out-Of-Distribution Detection: A Model-Specific Perspective (ICCV 2023) [paper]
Revisit PCA-based technique for Out-of-Distribution Detection (ICCV 2023) [paper] [code]
Hierarchical Visual Categories Modeling: A Joint Representation Learning and Density Estimation Framework for Out-of-Distribution Detection (ICCV 2023) [paper]
Out-of-Distribution Detection for Monocular Depth Estimation (ICCV 2023) [paper]
Simple and Effective Out-of-Distribution Detection via Cosine-based Softmax Loss (ICCV 2023) [paper]
Unsupervised Out-of-Distribution Detection with Diffusion Inpainting (ICML 2023) [paper]
Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection (ICML 2023) [paper] [code]
In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation (ICML 2023) [paper] [code]
Hybrid Energy Based Model in the Feature Space for Out-of-Distribution Detection (ICML 2023) [paper] [code]
Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability (ICML 2023) [paper] [code]
Detecting Out-of-distribution Data through In-distribution Class Prior (ICML 2023) [paper] [code]
Concept-based Explanations for Out-of-Distribution Detectors (ICML 2023) [paper] [code]
Decoupling MaxLogit for Out-of-Distribution Detection (CVPR 2023) [paper] [code]
Block Selection Method for Using Feature Norm in Out-of-Distribution Detection (CVPR 2023) [paper] [code]
LINe: Out-of-Distribution Detection by Leveraging Important Neurons (CVPR 2023) [paper] [code]
Uncertainty-Aware Optimal Transport for Semantically Coherent Out-of-Distribution Detection (CVPR 2023) [paper] [code]
Rethinking Out-of-Distribution (OOD) Detection: Masked Image Modeling Is All You Need (CVPR 2023) [paper] [code]
Balanced Energy Regularization Loss for Out-of-Distribution Detection (CVPR 2023) [paper]
Detection of Out-of-Distribution Samples Using Binary Neuron Activation Patterns (CVPR 2023) [paper] [code]
GEN: Pushing the Limits of Softmax-Based Out-of-Distribution Detection (CVPR 2023) [paper] [code]
OpenMix: Exploring Outlier Samples for Misclassification Detection (CVPR 2023) [paper] [code]
Packed-Ensembles for Efficient Uncertainty Estimation (ICLR 2023) [paper] [code]
A framework for benchmarking Class-out-of-distribution detection and its application to ImageNet (ICLR 2023) [paper] [code]
Out-of-Distribution Detection based on In-Distribution Data Patterns Memorization with Modern Hopfield Energy (ICLR 2023) [paper] [code]
Extremely Simple Activation Shaping for Out-of-Distribution Detection (ICLR 2023) [paper] [code]
The Tilted Variational Autoencoder: Improving Out-of-Distribution Detection (ICLR 2023) [paper]
Out-of-distribution Detection with Implicit Outlier Transformation (ICLR 2023) [paper] [code]
Energy-based Out-of-Distribution Detection for Graph Neural Networks (ICLR 2023) [paper] [code]
How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection? (ICLR 2023) [paper] [code]
Harnessing Out-Of-Distribution Examples via Augmenting Content and Style (ICLR 2023) [paper]
Fake It Until You Make It : Towards Accurate Near-Distribution Novelty Detection (ICLR 2023) [paper] [code]
Non-parametric Outlier Synthesis (ICLR 2023) [paper] [code]
Out-of-Distribution Detection and Selective Generation for Conditional Language Models (ICLR 2023) [paper]
Turning the Curse of Heterogeneity in Federated Learning into a Blessing for Out-of-Distribution Detection (ICLR 2023) [paper] [code]

2022

Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD Training Data Estimate a Combination of the Same Core Quantities (ICML 2022) [paper] [code]
POEM: Out-of-Distribution Detection with Posterior Sampling (ICML 2022) [paper] [code]
Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognition (ICML 2022) [paper] [code]
Scaling Out-of-Distribution Detection for Real-World Settings (ICML 2022) [paper] [code]
Mitigating Neural Network Overconfidence with Logit Normalization (ICML 2022) [paper] [code]
Out-of-Distribution Detection with Deep Nearest Neighbors (ICML 2022) [paper] [code]
Training OOD Detectors in their Natural Habitats (ICML 2022) [paper] [code]
Out-of-Distribution Detection via Conditional Kernel Independence Model (NeurIPS 2022) [paper] [code]
Your Out-of-Distribution Detection Method is Not Robust! (NeurIPS 2022) [paper] [code]
Boosting Out-of-distribution Detection with Typical Features (NeurIPS 2022) [paper] [code]
RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection (NeurIPS 2022) [paper]
Provably Adversarially Robust Detection of Out-of-Distribution Data (Almost) for Free (NeurIPS 2022) [paper] [code]
Watermarking for Out-of-distribution Detection (NeurIPS 2022) [paper] [code]
Is Out-of-Distribution Detection Learnable? (NeurIPS 2022) [paper]
RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy & Out-of-Distribution Robustness (NeurIPS 2022) [paper] [code]
Out-of-Distribution Detection with An Adaptive Likelihood Ratio on Informative Hierarchical VAE (NeurIPS 2022) [paper]
GraphDE: A Generative Framework for Debiased Learning and Out-of-Distribution Detection on Graphs (NeurIPS 2022) [paper] [code]
Density-driven Regularization for Out-of-distribution Detection (NeurIPS 2022) [paper]
Delving into Out-of-Distribution Detection with Vision-Language Representations (NeurIPS 2022) [paper] [code]
Beyond Mahalanobis Distance for Textual OOD Detection (NeurIPS 2022) [paper]
SIREN: Shaping Representations for Detecting Out-of-Distribution Objects (NeurIPS 2022) [paper] [code]
Out-of-distribution Detection with Boundary Aware Learning (ECCV 2022) [paper]
Out-of-Distribution Detection with Semantic Mismatch under Masking (ECCV 2022) [paper] [code]
Data Invariants to Understand Unsupervised Out-of-Distribution Detection (ECCV 2022) [paper]
Out-of-Distribution Identification: Let Detector Tell Which I Am Not Sure (ECCV 2022) [paper]
Deep Hybrid Models for Out-of-Distribution Detection (CVPR 2022) [paper]
Rethinking Reconstruction Autoencoder-Based Out-of-Distribution Detection (CVPR 2022) [paper]
ViM: Out-of-Distribution With Virtual-Logit Matching (CVPR 2022) [paper] [code]
Neural Mean Discrepancy for Efficient Out-of-Distribution Detection (CVPR 2022) [paper]
Unknown-Aware Object Detection: Learning What You Don’t Know from Videos in the Wild (CVPR 2022) [paper] [code]
Revisiting flow generative models for Out-of-distribution detection (ICLR 2022) [paper]
Igeood: An Information Geometry Approach to Out-of-Distribution Detection (ICLR 2022) [paper]
A Statistical Framework for Efficient Out of Distribution Detection in Deep Neural Networks (ICLR 2022) [paper]
VOS: Learning What You Don't Know by Virtual Outlier Synthesis (ICLR 2022) [paper] [code]

2021

Understanding Failures in Out-of-Distribution Detection with Deep Generative Models (ICML 2021) [paper]
Exploring the Limits of Out-of-Distribution Detection (NeurIPS 2021) [paper]
STEP: Out-of-Distribution Detection in the Presence of Limited In-Distribution Labeled Data (NeurIPS 2021) [paper]
Locally Most Powerful Bayesian Test for Out-of-Distribution Detection Using Deep Generative Models (NeurIPS 2021) [paper]
Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection (NeurIPS 2021) [paper]
ReAct: Out-of-distribution Detection With Rectified Activations (NeurIPS 2021) [paper] [code]
Entropy Maximization and Meta Classification for Out-of-Distribution Detection in Semantic Segmentation (ICCV 2021) [paper]
Semantically Coherent Out-of-Distribution Detection (ICCV 2021) [paper] [code]
Triggering Failures: Out-of-Distribution Detection by Learning From Local Adversarial Attacks in Semantic Segmentation (ICCV 2021) [paper] [code]
MOOD: Multi-Level Out-of-Distribution Detection (CVPR 2021) [paper]
MOS: Towards Scaling Out-of-Distribution Detection for Large Semantic Space (CVPR 2021) [paper]
Out-of-Distribution Detection Using Union of 1-Dimensional Subspaces (CVPR 2021) [paper] [code]
Multiscale Score Matching for Out-of-Distribution Detection (ICLR 2021) [paper] [code]
SSD: A Unified Framework for Self-Supervised Outlier Detection (ICLR 2021) [paper] [code]

Early Works

Energy-based Out-of-distribution Detection (NeurIPS 2020) [paper] [code]
CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances (NeurIPS 2020) [paper] [code]
Why Normalizing Flows Fail to Detect Out-of-Distribution Data (NeurIPS 2020) [paper] [code]
Likelihood Ratios for Out-of-Distribution Detection (NeurIPS 2019) [paper]
Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem (CVPR 2019) [paper]
Deep Anomaly Detection with Outlier Exposure (ICLR 2019) [paper] [code]
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks (NeurIPS 2018) [paper] [code]
Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks (ICLR 2018) [paper]
Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples (ICLR 2018) [paper] [code]
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (NeurIPS 2018) [paper]
Predictive Uncertainty Estimation via Prior Networks (NeurIPS 2018) [paper] [code]
Out-of-Distribution Detection using Multiple Semantic Label Representations (NeurIPS 2018) [paper]
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks (ICLR 2017) [paper]

Impression2805 / Awesome-Failure-Detection