pyprob is a PyTorch-based library for probabilistic programming and inference compilation.
Inference compilation is a technique for performing fast inference in generative models implemented as probabilistic programs, using deep neural networks to parameterize proposal distributions of a sequential importance sampling inference engine.
- Python 3.5 or higher. We recommend Anaconda.
- Latest PyTorch, installed by following instructions on the PyTorch web site.
To use a cutting-edge version, clone this repository and install the pyprob package using:
git clone git@github.com:probprog/pyprob.git
cd pyprob
pip install .
To use the latest version available in Python Package Index, run:
pip install pyprob
pyprob has two main modes of operation:
- Probabilistic programming and inference compilation fully in Python
- Interfacing with 3rd party probabilistic programming libraries (e.g., Anglican, CPProb) through a ZeroMQ/FlatBuffers-based protocol
NOTE: This is currently a work in progress, and the code in this public repository does not represent a fully-usable system. A website with documentation and examples will be provided in due course.
After setting up the probabilistic program and initiating compilation mode in the 3rd party library, you can start the compilation module with the default set of parameters using:
pyprob-compile
This starts a training session with infinite training data supplied by the probabilistic model. You can stop training at any point by hitting Ctrl + C. Alternatively, you can use the --maxTraces option to stop after a set number of traces (e.g., --maxTraces 1000).
If you want to use GPU, you should install PyTorch with CUDA support and use the --cuda flag.
By default the compilation artifacts are saved to the current directory. This can be changed by using the --dir option (e.g., --dir ~/artifacts).
There are a number of parameters for configuring the compilation session, such as setting different embedding types and neural network architectures. For information on the various command line options available for compilation, use the --help flag.
After setting up the probabilistic program and initiating inference mode in the 3rd party library, you can start the inference module with the default set of parameters using:
pyprob-infer
This starts an inference session using the latest saved artifact in the current directory. The directory for loading the artifact from can be changed using the --dir option (e.g., --dir ~/artifacts). Inference can be run on GPU using the --cuda flag.
Use the --help flag to see all available options and functionality.
You can use
pyprob-analytics
for showing various statistics about the latest saved artifact in the current directory. You can use the --help flag to see available options for other functionality including the production of loss plots and detailed analytics reports.
Our paper at AISTATS 2017 provides an in-depth description of the inference compilation technique.
If you use pyprob and/or would like to cite our paper, please use the following information:
@inproceedings{le-2016-inference,
author = {Le, Tuan Anh and Baydin, Atılım Güneş and Wood, Frank},
booktitle = {Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS)},
title = {Inference Compilation and Universal Probabilistic Programming},
year = {2017},
volume = {54},
pages = {1338--1348},
series = {Proceedings of Machine Learning Research},
address = {Fort Lauderdale, FL, USA},
publisher = {PMLR}
}
pyprob is distributed under the MIT License.
pyprob has been developed by Tuan Anh Le and Atılım Güneş Baydin within the Probabilistic Programming Group at the University of Oxford, led by Frank Wood.
For the full list of contributors, see: