InvertibleNetworks.jl

Building blocks for invertible neural networks in the Julia programming language.

Installation

] dev https://github.com/slimgroup/InvertibleNetworks.jl

1x1 Convolutions using Householder transformations (example)
Residual block (example)
Invertible coupling layer from Dinh et al. (2017) (example)
Invertible hyperbolic layer from Lensink et al. (2019) (example)
Invertible coupling layer from Putzky and Welling (2019) (example)
Invertible recursive coupling layer HINT from Kruse et al. (2020) (example)
Activation normalization (Kingma and Dhariwal, 2018) (example)
Various activation functions (Sigmoid, ReLU, leaky ReLU, GaLU)
Objective and misfit functions (mean squared error, log-likelihood)
Dimensionality manipulation: squeeze/unsqueeze (column, patch, checkerboard), split/cat
Squeeze/unsqueeze using the wavelet transform

Invertible recurrent inference machines (Putzky and Welling, 2019) (generic example)
Generative models with maximum likelihood via the change of variable formula (example)
Glow: Generative flow with invertible 1x1 convolutions (Kingma and Dhariwal, 2018) (generic example, source)

This package uses functions from NNlib.jl, Flux.jl and Wavelets.jl

Yann Dauphin, Angela Fan, Michael Auli and David Grangier, "Language modeling with gated convolutional networks", Proceedings of the 34th International Conference on Machine Learning, 2017. https://arxiv.org/pdf/1612.08083.pdf
Laurent Dinh, Jascha Sohl-Dickstein and Samy Bengio, "Density estimation using Real NVP", International Conference on Learning Representations, 2017, https://arxiv.org/abs/1605.08803
Diederik P. Kingma and Prafulla Dhariwal, "Glow: Generative Flow with Invertible 1x1 Convolutions", Conference on Neural Information Processing Systems, 2018. https://arxiv.org/abs/1807.03039
Keegan Lensink, Eldad Haber and Bas Peters, "Fully Hyperbolic Convolutional Neural Networks", arXiv Computer Vision and Pattern Recognition, 2019. https://arxiv.org/abs/1905.10484
Patrick Putzky and Max Welling, "Invert to learn to invert", Advances in Neural Information Processing Systems, 2019. https://arxiv.org/abs/1911.10914
Jakob Kruse, Gianluca Detommaso, Robert Scheichl and Ullrich Köthe, "HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inference", arXiv Statistics and Machine Learning, 2020. https://arxiv.org/abs/1905.10687