ConanCui / Bayesian-deep-learning

Notes and codes of the topic "Bayesian deep learning"

Bayesian-deep-learning

Notes and codes of the topic "Bayesian deep learning"

References (continually updated) :

Preliminaries

• A history of Bayesian neural network, https://www.youtube.com/watch?v=FD8l2vPU5FY
• Ghahramani, Zoubin. "Probabilistic machine learning and artificial intelligence." Nature 521.7553 (2015): 452.

Variational inference

• Graves, Alex. "Practical variational inference for neural networks." Advances in neural information processing systems. 2011.
• Blundell, Charles, et al. "Weight uncertainty in neural networks." arXiv preprint arXiv:1505.05424 (2015).
• Rezende, Danilo Jimenez, and Shakir Mohamed. "Variational inference with normalizing flows." arXiv preprint arXiv:1505.05770 (2015).
• Louizos, Christos, and Max Welling. "Multiplicative normalizing flows for variational bayesian neural networks." arXiv preprint arXiv:1703.01961 (2017).
• Kingma, Diederik P., et al. "Improved variational inference with inverse autoregressive flow." Advances in Neural Information Processing Systems. 2016.
• Hernández-Lobato, José Miguel, et al. "Black-box α-divergence minimization." (2016).
• Li, Yingzhen, and Richard E. Turner. "Rényi divergence variational inference." Advances in Neural Information Processing Systems. 2016.

Dropout and Dropout network

• Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." The Journal of Machine Learning Research 15.1 (2014): 1929-1958.
• Kingma, Diederik P., Tim Salimans, and Max Welling. "Variational dropout and the local reparameterization trick." Advances in Neural Information Processing Systems. 2015.
• Gal, Yarin, and Zoubin Ghahramani. "Bayesian convolutional neural networks with Bernoulli approximate variational inference." arXiv preprint arXiv:1506.02158 (2015).
• Gal, Yarin, and Zoubin Ghahramani. "Dropout as a Bayesian approximation: Representing model uncertainty in deep learning." international conference on machine learning. 2016.
• Gal, Yarin, and Zoubin Ghahramani. "A theoretically grounded application of dropout in recurrent neural networks." Advances in neural information processing systems. 2016.
• Gal, Yarin, Jiri Hron, and Alex Kendall. "Concrete dropout." Advances in Neural Information Processing Systems. 2017.
• Li, Yingzhen, and Yarin Gal. "Dropout inference in bayesian neural networks with alpha-divergences." arXiv preprint arXiv:1703.02914 (2017).
• Gomez, A. N., Zhang, I., Swersky, K., Gal, Y., & Hinton, G. E. (2018). Targeted Dropout.

Other posterior approximation methods

Uncertainty estimation using Bayesian's eye

• Der Kiureghian, Armen, and Ove Ditlevsen. "Aleatory or epistemic? Does it matter?." Structural Safety 31.2 (2009): 105-112.
• Gal, Yarin. "Uncertainty in deep learning." University of Cambridge (2016).
• Kendall, Alex, Vijay Badrinarayanan, and Roberto Cipolla. "Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding." arXiv preprint arXiv:1511.02680 (2015).
• Kendall, Alex, and Yarin Gal. "What uncertainties do we need in bayesian deep learning for computer vision?." Advances in neural information processing systems. 2017.
• Kendall, Alex, Yarin Gal, and Roberto Cipolla. "Multi-task learning using uncertainty to weigh losses for scene geometry and semantics." arXiv preprint arXiv:1705.07115 3 (2017).

Uncertainty-aware Deep Learning

• Heo, Jay, et al. "Uncertainty-Aware Attention for Reliable Interpretation and Prediction." arXiv preprint arXiv:1805.09653 (2018).

Bayesian active learning

• Cohn, David A., Zoubin Ghahramani, and Michael I. Jordan. "Active learning with statistical models." Journal of artificial intelligence research 4 (1996): 129-145.
• Gal, Yarin, Riashat Islam, and Zoubin Ghahramani. "Deep bayesian active learning with image data." arXiv preprint arXiv:1703.02910 (2017).
• Deep Bayesian Active Learning with Image Data, https://vimeo.com/240606680
• Rottmann, Matthias, Karsten Kahl, and Hanno Gottschalk. "Deep Bayesian Active Semi-Supervised Learning." arXiv preprint arXiv:1803.01216 (2018).
• Chen, Nutan, et al. "Active learning based on data uncertainty and model sensitivity." arXiv preprint arXiv:1808.02026 (2018).

Adversarial examples

• Gal, Yarin, and Lewis Smith. "Sufficient conditions for idealised models to have no adversarial examples: a theoretical and empirical study with Bayesian neural networks." arXiv preprint arXiv:1806.00667 (2018).