NeoChaos12 / BNNBench

Benchmarking framework for Bayesian Neural Networks as surrogate models for Bayesian Optimization.

BNNBench

Bayesian Neural Network Benchmarker for Bayesian Optimization.

This repository contains a flexible framework for benchmarking the performance of BNNs as surrogate models for Bayesian Optimization. The main modules are:

• models: A collection of BNNs. See below for a list of available models.
• emukit_interfaces: A collection of wrappers and helper modules to run the BO benchmarking with the help of emukit.
• postprocessing: Data collection, collation and post-processing module. All collected data will be stored as Pandas dataframes.
• visualization: Data visualization modules, designed for (almost) arbitrarily structured Pandas DataFrames.

The following models have been implemented:

• MC-DropOut (MCDO)
• MC-BatchNorm (MCBN)
• Deep Ensemble
• Scalable Bayesian Optimization Using Deep Neural Networks (DNGO) - Adapted from this implementation of DNGO.

Installation

BNNBench can be installed by typing the following series of commands in a terminal on linux:

git clone https://github.com/NeoChaos12/BNNBench.git
cd bnnbench
python setup.py install

Alternatively, after downloading the repo in a directory and ensuring all the dependencies are installed, set the environment variable "BNNBENCHPATH" to the full-path of the directory where the code was downloaded. Most of the useful run-scripts will detect this path and still work without installing the repository.