Awesome Uncertainty in Deep learning
This repo is a collection of AWESOME papers, codes, books, and blogs about Uncertainty and Deep learning. Feel free to star and fork.
If you think that we miss a paper, or if you have any ideas for improvements, please send a message on the corresponding GitHub discussions.
You may also send an email at:
gianni dot franchi at ensta-paris dot fr
with "[Awesome Uncertainty]" as subject. Tell us where the paper was published and where, and send us GitHub and ArXiv links if they are available.
Table of Contents
- Awesome Uncertainty in Deep learning
- Papers
- Surveys
- Theory
- Bayesian-Methods
- Ensemble-Methods
- Sampling/Dropout-based-Methods
- Auxiliary-Methods/Learning-loss-distributions
- Data-augmentation/Generation-based-methods
- Dirichlet-networks/Evidential-deep-learning
- Deterministic-Uncertainty-Methods
- Quantile-Regression/Predicted-Intervals
- Calibration
- Applications
- Datasets and Benchmarks
- Libraries
- Lectures and tutorials
- Books
- Other resources
Papers
Surveys
Conference
- A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications[AISafety Workshop2020]
Journal
- Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods [Machine Learning]
- Predictive inference with the jackknife+ [The Annals of Statistics (2021)]
- A review of uncertainty quantification in deep learning: Techniques, applications and challenges [Information Fusion 2021]
- A Survey on Uncertainty Estimation in Deep Learning Classification Systems from a Bayesian Perspective [ACM2021]
- Uncertainty in big data analytics: survey, opportunities, and challenges [Journal of Big Data2019]
Arxiv
- A Survey on Uncertainty Reasoning and Quantification for Decision Making: Belief Theory Meets Deep Learning [arXiv2022]
- Ensemble deep learning: A review [arXiv2021]
- A survey of uncertainty in deep neural networks [arXiv2021] - [GitHub]
- A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation [arXiv2021]
Theory
Conference
- Top-label calibration and multiclass-to-binary reductions [ICLR2022]
- Neural Variational Gradient Descent [AABI2022]
- Repulsive Deep Ensembles are Bayesian [NeurIPS2021] - [PyTorch]
- Bayesian Optimization with High-Dimensional Outputs [NeurIPS2021]
- Residual Pathway Priors for Soft Equivariance Constraints [NeurIPS2021]
- Dangers of Bayesian Model Averaging under Covariate Shift [NeurIPS2021] - [TensorFlow]
- A Mathematical Analysis of Learning Loss for Active Learning in Regression [CVPR Workshop2021]
- Uncertainty in Gradient Boosting via Ensembles [ICLR2021] - [PyTorch]
- Deep Convolutional Networks as shallow Gaussian Processes [ICLR2019]
- On the accuracy of influence functions for measuring group effects [NeurIPS2018]
- To Trust Or Not To Trust A Classifier [NeurIPS2018] - [Python]
- Understanding Measures of Uncertainty for Adversarial Example Detection [UAI2018]
Journal
- Multivariate Uncertainty in Deep Learning [TNNLS2021]
- A General Framework for Uncertainty Estimation in Deep Learning [RAL2020]
- Adaptive nonparametric confidence sets [The Annals of Statistic (2006)]
Arxiv
- Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping [arXiv2022]
- Bayesian Model Selection, the Marginal Likelihood, and Generalization [arXiv2022]
- Testing for Outliers with Conformal p-values [arXiv2021] - [Python]
- Efficient Gaussian Neural Processes for Regression [arXiv2021]
- Dense Uncertainty Estimation [arXiv2021] - [PyTorch]
- Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning [arXiv2020]
- A higher-order swiss army infinitesimal jackknife [arXiv2019]
- With malice towards none: Assessing uncertainty via equalized coverage [arXiv2019]
Bayesian-Methods
Conference
- Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture [NeurIPS2022]
- Activation-level uncertainty in deep neural networks [ICLR2021]
- On the Effects of Quantisation on Model Uncertainty in Bayesian Neural Networks [UAI2021]
- Learnable uncertainty under Laplace approximations [UAI2021]
- Bayesian Deep Learning and a Probabilistic Perspective of Generalization [NeurIPS2020]
- How Good is the Bayes Posterior in Deep Neural Networks Really? [ICML2020]
- Bayesian Uncertainty Estimation for Batch Normalized Deep Networks [ICML2020]
- Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors [ICML2020] - [TensorFlow]
- Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks [ICML2020]
- TRADI: Tracking deep neural network weight distributions for uncertainty estimation [ECCV2020] - [PyTorch]
- A Simple Baseline for Bayesian Uncertainty in Deep Learning [NeurIPS2019] - [PyTorch]
- Lightweight Probabilistic Deep Networks [CVPR2018] - [PyTorch]
- A Scalable Laplace Approximation for Neural Networks [ICLR2018] - [Theano]
- Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning [ICML2018]
- Weight Uncertainty in Neural Network [ICML2015]
Journal
- Bayesian modeling of uncertainty in low-level vision [IJCV1990]
Arxiv
- Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantification [arXiv2020] - [PyTorch]
- Bayesian Neural Networks with Soft Evidence [arXiv2020] - [PyTorch]
- On Batch Normalisation for Approximate Bayesian Inference [arXiv2020]
- Bayesian neural network via stochastic gradient descent [arXiv2020]
Ensemble-Methods
Conference
- Normalizing Flow Ensembles for Rich Aleatoric and Epistemic Uncertainty Modeling [AAAI2023]
- Packed-Ensembles for Efficient Uncertainty Estimation [ICLR2023] - [PyTorch]
- FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation [NeurIPS2022]
- Prune and Tune Ensembles: Low-Cost Ensemble Learning With Sparse Independent Subnetworks [AAAI2022]
- Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity [ICLR2022] - [PyTorch]
- Robustness via Cross-Domain Ensembles [ICCV2021] - [PyTorch]
- Masksembles for Uncertainty Estimation [CVPR2021] - [PyTorch/TensorFlow]
- Uncertainty Quantification and Deep Ensembles [NeurIPS2021]
- Uncertainty in Gradient Boosting via Ensembles [ICLR2021] - [PyTorch]
- Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning [ICLR2020] - [PyTorch]
- Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles [AAAI2020]
- Hyperparameter Ensembles for Robustness and Uncertainty Quantification [NeurIPS2020]
- Bayesian Deep Ensembles via the Neural Tangent Kernel [NeurIPS2020]
- BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning [ICLR2020] - [TensorFlow] - [PyTorch]
- Uncertainty in Neural Networks: Approximately Bayesian Ensembling [AISTATS 2020]
- Accurate Uncertainty Estimation and Decomposition in Ensemble Learning [NeurIPS2019]
- High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach [ICML2018] - [TensorFlow]
- Simple and scalable predictive uncertainty estimation using deep ensembles [NeurIPS2017]
Journal
- One Versus all for deep Neural Network for uncertaInty (OVNNI) quantification [IEEE Access2021]
Arxiv
- Deep Ensembles Work, But Are They Necessary? [arXiv2022]
- On the Usefulness of Deep Ensemble Diversity for Out-of-Distribution Detection [arXiv2022]
- Deep Ensemble as a Gaussian Process Approximate Posterior [arXiv2022]
- Sequential Bayesian Neural Subnetwork Ensembles [arXiv2022]
- Confident Neural Network Regression with Bootstrapped Deep Ensembles [arXiv2022] - [TensorFlow]
- Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [arXiv2021]
- Deep Ensembles: A Loss Landscape Perspective [arXiv2019]
- Diversity with Cooperation: Ensemble Methods for Few-Shot Classification [arXiv2019]
Sampling/Dropout-based-Methods
Conference
- Efficient Bayesian Uncertainty Estimation for nnU-Net [MICCAI2022]
- Training-Free Uncertainty Estimation for Dense Regression: Sensitivity as a Surrogate [AAAI2022]
- Dropout Sampling for Robust Object Detection in Open-Set Conditions [ICRA2018]
- Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks [MIDL2018]
- Concrete Dropout [NeurIPS2017]
- Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning [ICML2016]
Journal
- A General Framework for Uncertainty Estimation in Deep Learning [Robotics and Automation Letters2020]
Arxiv
- SoftDropConnect (SDC) – Effective and Efficient Quantification of the Network Uncertainty in Deep MR Image Analysis [arXiv2022]
- Wasserstein Dropout [arXiv2021] - [PyTorch]
Auxiliary-Methods/Learning-loss-distributions
Conference
- Post-hoc Uncertainty Learning using a Dirichlet Meta-Model [AAAI2023]
- Improving the reliability for confidence estimation [ECCV2022]
- Gradient-based Uncertainty for Monocular Depth Estimation [ECCV2022] - [PyTorch]
- BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks [ECCV2022] - [PyTorch]
- Detecting Misclassification Errors in Neural Networks with a Gaussian Process Model [AAAI2022]
- Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation [NeurIPS2022]
- Learning Structured Gaussians to Approximate Deep Ensembles [CVPR2022]
- SLURP: Side Learning Uncertainty for Regression Problems [BMVC2021] - [PyTorch]
- Learning to Predict Error for MRI Reconstruction [MICCAI2021]
- Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation [ICCV2021] - [PyTorch]
- A Mathematical Analysis of Learning Loss for Active Learning in Regression [CVPR Workshop2021]
- Real-time uncertainty estimation in computer vision via uncertainty-aware distribution distillation [WACV2021]
- Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel [ICLR2020] - [TensorFlow]
- Gradients as a Measure of Uncertainty in Neural Networks [ICIP2020]
- Learning Loss for Test-Time Augmentation [NeurIPS2020]
- On the uncertainty of self-supervised monocular depth estimation [CVPR2020] - [PyTorch]
- Addressing failure prediction by learning model confidence [NeurIPS2019] - [PyTorch]
- Learning loss for active learning [CVPR2019] - [PyTorch] (unofficial codes)
- Structured Uncertainty Prediction Networks [CVPR2018] - [TensorFlow]
- Uncertainty estimates and multi-hypotheses networks for optical flow [ECCV2018] - [TensorFlow]
- Classification uncertainty of deep neural networks based on gradient information [IAPR Workshop2018]
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? [NeurIPS2017]
- Estimating the Mean and Variance of the Target Probability Distribution [(ICNN94)]
Journal
- Confidence Estimation via Auxiliary Models [TPAMI2021]
Arxiv
- Instance-Aware Observer Network for Out-of-Distribution Object Segmentation [arXiv2022]
- Learning Uncertainty For Safety-Oriented Semantic Segmentation In Autonomous Driving [arXiv2022]
- DEUP: Direct Epistemic Uncertainty Prediction [arXiv2020]
- Learning Confidence for Out-of-Distribution Detection in Neural Networks [arXiv2018]
Data-augmentation/Generation-based-methods
Conference
- Diverse, Global and Amortised Counterfactual Explanations for Uncertainty Estimates [AAAI2022]
- PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures [CVPR2022]
- Towards efficient feature sharing in MIMO architectures [CVPR Workshop2022]
- Robust Semantic Segmentation with Superpixel-Mix [BMVC2021] - [PyTorch]
- MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks [ICCV2021] - [PyTorch]
- Training independent subnetworks for robust prediction [ICLR2021]
- Uncertainty-aware GAN with Adaptive Loss for Robust MRI Image Enhancement [ICCV Workshop2021]
- Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning [ICML2020]
- Uncertainty-Aware Deep Classifiers using Generative Models [AAAI2020]
- Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation [ECCV2020] - [PyTorch]
- Detecting the Unexpected via Image Resynthesis [ICCV2019] - [PyTorch]
- On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks [NeurIPS2019]
- Deep Anomaly Detection with Outlier Exposure [ICLR2019]
Arxiv
- ZigZag: Universal Sampling-free Uncertainty Estimation Through Two-Step Inference [arXiv2022]
- Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [arXiv2021]
- Quantifying uncertainty with GAN-based priors [arXiv2019]
Dirichlet-networks/Evidential-deep-learning
Conference
- An Evidential Neural Network Model for Regression Based on Random Fuzzy Numbers [BELIEF2022]
- Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions [ICLR2022] - [PyTorch]
- TBraTS: Trusted Brain Tumor Segmentation [MICCAI2022]
- Improving Evidential Deep Learning via Multi-task Learning [AAAI2022]
- Trustworthy multimodal regression with mixture of normal-inverse gamma distributions [NeurIPS2021]
- Misclassification Risk and Uncertainty Quantification in Deep Classifiers [WACV2021]
- Evaluating robustness of predictive uncertainty estimation: Are Dirichlet-based models reliable? [ICML2021]
- Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts [NeurIPS2020] - [PyTorch]
- Being Bayesian about Categorical Probability [ICML2020]
- Ensemble Distribution Distillation [ICLR2020]
- Conservative Uncertainty Estimation By Fitting Prior Networks [ICLR2020]
- Noise Contrastive Priors for Functional Uncertainty [UAI2020]
- Deep Evidential Regression [NeurIPS2020] - [TensorFlow]
- Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examples [NeurIPS Workshop2020]
- Uncertainty on Asynchronous Time Event Prediction [NeurIPS2019] - [TensorFlow]
- Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness [NeurIPS2019]
- Quantifying Classification Uncertainty using Regularized Evidential Neural Networks [AAAI FSS2019]
- Evidential Deep Learning to Quantify Classification Uncertainty [NeurIPS2018] - [PyTorch]
- Predictive uncertainty estimation via prior networks [NeurIPS2018]
Journal
- Region-Based Evidential Deep Learning to Quantify Uncertainty and Improve Robustness of Brain Tumor Segmentation [NCA2022]
- An evidential classifier based on Dempster-Shafer theory and deep learning [Neurocomputing2021] - [TensorFlow]
- Evidential fully convolutional network for semantic segmentation [AppliedIntelligence2021] - [TensorFlow]
- Information Aware max-norm Dirichlet networks for predictive uncertainty estimation [NeuralNetworks2021]
- A neural network classifier based on Dempster-Shafer theory [IEEETransSMC2000]
Arxiv
- Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [arXiv2023]
- The Unreasonable Effectiveness of Deep Evidential Regression [arXiv2022]
- Effective Uncertainty Estimation with Evidential Models for Open-World Recognition [arXiv2022]
- Multivariate Deep Evidential Regression [arXiv2022]
- A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation [arXiv2021]
- Regression Prior Networks [arXiv2020]
- Uncertainty estimation in deep learning with application to spoken language assessment[PhDThesis2019]
- Inhibited softmax for uncertainty estimation in neural networks [arXiv2018].
- Quantifying Intrinsic Uncertainty in Classification via Deep Dirichlet Mixture Networks [arXiv2018]
Deterministic-Uncertainty-Methods
Conference
- Training, Architecture, and Prior for Deterministic Uncertainty Methods [ICLR Workshop2023] - [PyTorch]
- Latent Discriminant deterministic Uncertainty [ECCV2022] - [PyTorch]
- Improving Deterministic Uncertainty Estimation in Deep Learning for Classification and Regression [CoRR2021]
- Training normalizing flows with the information bottleneck for competitive generative classification [NeurIPS2020]
- Simple and principled uncertainty estimation with deterministic deep learning via distance awareness [NeurIPS2020]
- Uncertainty Estimation Using a Single Deep Deterministic Neural Network [ICML2020] - [PyTorch]
- Single-Model Uncertainties for Deep Learning [NeurIPS2019] - [PyTorch]
- Sampling-Free Epistemic Uncertainty Estimation Using Approximated Variance Propagation [ICCV2019] - [PyTorch]
Journal
- Density estimation in representation space [EDSMLS2020]
Arxiv
- Deep Deterministic Uncertainty: A Simple Baseline [arXiv2021] - [PyTorch]
- Deep Deterministic Uncertainty for Semantic Segmentation [arXiv2021]
- On the Practicality of Deterministic Epistemic Uncertainty [arXiv2021]
- The Hidden Uncertainty in a Neural Network’s Activations [arXiv2020]
- A simple framework for uncertainty in contrastive learning [arXiv2020]
- Distance-based Confidence Score for Neural Network Classifiers [arXiv2017]
Quantile-Regression/Predicted-Intervals
Conference
- Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging [ICML2022] - [PyTorch]
- Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles [UAI2020] - [PyTorch]
- Classification with Valid and Adaptive Coverage [NeurIPS2020]
- Conformal Prediction Under Covariate Shift [NeurIPS2019]
- Conformalized Quantile Regression [NeurIPS2019]
- Single-Model Uncertainties for Deep Learning [NeurIPS2019] - [PyTorch]
- High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach [ICML2018] - [TensorFlow]
Journal
- Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors [CMAME2022]
- Exploring uncertainty in regression neural networks for construction of prediction intervals [Neurocomputing2022]
Arxiv
- Testing for Outliers with Conformal p-values [arXiv2021] - [Python]
- Interval Neural Networks: Uncertainty Scores [arXiv2020]
- Tight Prediction Intervals Using Expanded Interval Minimization [arXiv2018]
Calibration
Conference
- Beyond calibration: estimating the grouping loss of modern neural networks [ICLR2023]
- Rethinking Confidence Calibration for Failure Prediction [ECCV2022] - [PyTorch]
- The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration [CVPR2022] - [PyTorch]
- Calibrating Deep Neural Networks by Pairwise Constraints [CVPR2022]
- Top-label calibration and multiclass-to-binary reductions [ICLR2022]
- Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error [ICML Workshop2021] - [PyTorch]
- Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification [NeurIPS2021]
- From label smoothing to label relaxation [AAAI2021]
- Calibrating Deep Neural Networks using Focal Loss [NeurIPS2020] - [PyTorch]
- Stationary activations for uncertainty calibration in deep learning [NeurIPS2020]
- Confidence-Aware Learning for Deep Neural Networks [ICML2020] - [PyTorch]
- Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning [ICML2020]
- Regularization via structural label smoothing [ICML2020]
- Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning [MIDL2020] - [PyTorch]
- Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision [CVPR Workshop2020] - [PyTorch]
- When does label smoothing help? [NeurIPS2019]
- Verified Uncertainty Calibration [NeurIPS2019]
- Generalized zero-shot learning with deep calibration network [NeurIPS2018]
- Measuring Calibration in Deep Learning [CVPR Workshop2019]
- Accurate Uncertainties for Deep Learning Using Calibrated Regression [ICML2018]
- On calibration of modern neural networks [ICML2017]
- On Fairness and Calibration [NeurIPS2017]
Journal
- Evaluating and Calibrating Uncertainty Prediction in Regression Tasks [Sensors2022]
- Calibrated Prediction Intervals for Neural Network Regressors [IEEE Access 2018] - [Python]
Arxiv
- Towards Understanding Label Smoothing [arXiv2020]
- An Investigation of how Label Smoothing Affects Generalization [arXiv2020]
Applications
Classification and Semantic-Segmentation
Conference
- CRISP - Reliable Uncertainty Estimation for Medical Image Segmentation [MICCAI2022]
- TBraTS: Trusted Brain Tumor Segmentation [MICCAI2022] - [PyTorch]
- Anytime Dense Prediction with Confidence Adaptivity [ICLR2022] - [PyTorch]
- Robust Semantic Segmentation with Superpixel-Mix [BMVC2021] - [PyTorch]
- Classification with Valid and Adaptive Coverage [NeurIPS2020]
- DEAL: Difficulty-aware Active Learning for Semantic Segmentation [ACCV2020]
- Human Uncertainty Makes Classification More Robust [ICCV2019]
- Classification uncertainty of deep neural networks based on gradient information [IAPR Workshop2018]
- Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation [ICCV2019]
- Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation [MICCAI2019] - [PyTorch]
- A Probabilistic U-Net for Segmentation of Ambiguous Images [NeurIPS2018] - [PyTorch]
- Evidential Deep Learning to Quantify Classification Uncertainty [NeurIPS2018] - [PyTorch]
- Lightweight Probabilistic Deep Networks [CVPR2018] - [PyTorch]
- To Trust Or Not To Trust A Classifier [NeurIPS2018]
- Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding [BMVC2017]
Journal
- Explainable machine learning in image classification models: An uncertainty quantification perspective." [KnowledgeBased2022]
- Region-Based Evidential Deep Learning to Quantify Uncertainty and Improve Robustness of Brain Tumor Segmentation [NCA2022]
Arxiv
- Deep Deterministic Uncertainty for Semantic Segmentation [arXiv2021]
- Evaluating Bayesian Deep Learning Methods for Semantic Segmentation [arXiv2018]
Regression
Conference
- Variational Depth Networks: Uncertainty-Aware Monocular Self-supervised Depth Estimation [ECCV Workshop2022]
- Uncertainty Quantification in Depth Estimation via Constrained Ordinal Regression [ECCV2022]
- On Monocular Depth Estimation and Uncertainty Quantification using Classification Approaches for Regression [ICIP2022]
- Anytime Dense Prediction with Confidence Adaptivity [ICLR2022] - [PyTorch]
- Learning Structured Gaussians to Approximate Deep Ensembles [CVPR2022]
- Training-Free Uncertainty Estimation for Dense Regression: Sensitivity as a Surrogate [AAAI2022]
- Robustness via Cross-Domain Ensembles [ICCV2021] - [PyTorch]
- SLURP: Side Learning Uncertainty for Regression Problems [BMVC2021] - [PyTorch]
- Learning to Predict Error for MRI Reconstruction [MICCAI2021]
- Deep Evidential Regression [NeurIPS2020] - [TensorFlow]
- Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel [ICLR2020] - [TensorFlow]
- Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning [MIDL2020] - [PyTorch]
- On the uncertainty of self-supervised monocular depth estimation [CVPR2020] - [PyTorch]
- Fast Uncertainty Estimation for Deep Learning Based Optical Flow [IROS2020]
- Inferring Distributions Over Depth from a Single Image [IROS2019] - [TensorFlow]
- Multi-Task Learning based on Separable Formulation of Depth Estimation and its Uncertainty [CVPR Workshop2019]
- Lightweight Probabilistic Deep Networks [CVPR2018] - [PyTorch]
- Uncertainty estimates and multi-hypotheses networks for optical flow [ECCV2018] - [TensorFlow]
- Accurate Uncertainties for Deep Learning Using Calibrated Regression [ICML2018]
- Structured Uncertainty Prediction Networks [CVPR2018] - [TensorFlow]
Journal
- Evaluating and Calibrating Uncertainty Prediction in Regression Tasks [Sensors2022]
- Exploring uncertainty in regression neural networks for construction of prediction intervals [Neurocomputing2022]
- Deep Distribution Regression [Computational Statistics & Data Analysis2021]
- Calibrated Prediction Intervals for Neural Network Regressors [IEEE Access 2018] - [Python]
- Learning a Confidence Measure for Optical Flow [TPAMI2013]
Arxiv
- How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts? [arXiv2023] - [PyTorch]
- UncertaINR: Uncertainty Quantification of End-to-End Implicit Neural Representations for Computed Tomographaphy [arXiv2022]
- Efficient Gaussian Neural Processes for Regression [arXiv2021]
- Wasserstein Dropout [arXiv2021] - [PyTorch]
Anomaly-detection and Out-of-Distribution-Detection
Conference
- Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data [ACCV2022]
- Detecting Misclassification Errors in Neural Networks with a Gaussian Process Model [AAAI2022]
- VOS: Learning What You Don't Know by Virtual Outlier Synthesis [ICLR2022] - [PyTorch]
- Anomaly Detection via Reverse Distillation from One-Class Embedding [CVPR2022]
- Towards Total Recall in Industrial Anomaly Detection [CVPR22] - [PyTorch]
- Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection [WACV2022] - [PyTorch]
- On the Importance of Gradients for Detecting Distributional Shifts in the Wild [NeurIPS2021]
- Exploring the Limits of Out-of-Distribution Detection [NeurIPS2021]
- NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization [ICCV2021]
- NADS: Neural Architecture Distribution Search for Uncertainty Awareness [ICML2020]
- Energy-based Out-of-distribution Detection [NeurIPS2020]
- PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization [ICPR2020] - [PyTorch]
- Detecting out-of-distribution image without learning from out-of-distribution data. [CVPR2020]
- Learning Open Set Network with Discriminative Reciprocal Points [ECCV2020]
- Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation [ECCV2020] - [PyTorch]
- Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examples [NeurIPS Workshop2020]
- Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection [ICCV2019] - [PyTorch]
- Detecting the Unexpected via Image Resynthesis [ICCV2019] - [PyTorch]
- Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks [ICLR2018]
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks [ICLR2017] - [TensorFlow]
Journal
- One Versus all for deep Neural Network for uncertaInty (OVNNI) quantification [IEEE Access2021]
Arxiv
- Generalized out-of-distribution detection: A survey [arXiv2021]
- Do We Really Need to Learn Representations from In-domain Data for Outlier Detection? [arXiv2021]
- DATE: Detecting Anomalies in Text via Self-Supervision of Transformers [arXiv2021]
- Frequentist uncertainty estimates for deep learning [arXiv2018]
Object detection
Conference
- Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection [CVPR2023]
- Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors [ICLR2021]
- Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving [ICCV2019] - [CUDA] - [PyTorch] - [Keras]
Domain adaptation
Conference
- Uncertainty-guided Source-free Domain Adaptation [ECCV2022]
Datasets and Benchmarks
- SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation [CVPR2022]
- MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks [BMVC2022] - [PyTorch]
- ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding [ICCV2021]
- The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection [IJCV2021]
- SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation [NeurIPS2021]
- Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning [arXiv2021] - [TensorFlow]
- Curriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene Understanding [IJCV2020]
- Benchmarking the Robustness of Semantic Segmentation Models [CVPR2020]
- Fishyscapes: A Benchmark for Safe Semantic Segmentation in Autonomous Driving [ICCV Workshop2019]
- Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming [NeurIPS Workshop2019] - [GitHub]
- Semantic Foggy Scene Understanding with Synthetic Data [IJCV2018]
- Lost and Found: Detecting Small Road Hazards for Self-Driving Vehicles [IROS2016]
Libraries
- Fortuna [GitHub - JAX]
- Bayesian Torch [GitHub]
- A Bayesian Neural Network library for PyTorch [GitHub]
- TensorFlow Probability [Website]
- Uncertainty Toolbox [GitHub]
- Mixture Density Networks (MDN) for distribution and uncertainty estimation [GitHub]
- OpenOOD: Benchmarking Generalized OOD Detection [GitHub]
Lectures and tutorials
- Uncertainty and Robustness in Deep Learning Workshop in ICML (2020, 2021) [SlidesLive]
- Yarin Gal: BAYESIAN DEEP LEARNING 101 [website]
- MIT 6.S191: Evidential Deep Learning and Uncertainty (2021) [Youtube]
Books
- The "Probabilistic Machine-Learning" book series by Kevin Murphy [Book]
Other resources
Awesome conformal prediction [GitHub]
Uncertainty Quantification in Deep Learning [GitHub]
Awesome LLM Uncertainty Reliability Robustness [GitHub]