Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling
This is the official implementation of the paper Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling in Pytorch. We propose Transformer Neural Processes (TNPs), a new member of the Neural Processes family that casts uncertainty-aware meta learning as a sequence modeling problem. We learn TNPs via an autoregressive likelihood-based objective and instantiate it with a novel transformer-based architecture. TNPs achieve state-ofthe-art performance on various benchmark problems, outperforming all previous NP variants on meta regression, image completion, contextual multi-armed bandits, and Bayesian optimization.
Install
First, clone the repository:
git clone https://github.com/tung-nd/TNP-pytorch.git
Then install the dependencies as listed in env.yml and activate the environment:
conda env create -f env.yml
conda activate tnp
Usage
Please check the directory of each task for specific usage.
Citation
If you find this repo useful in your research, please consider citing our paper:
@article{nguyen2022transformer,
title={Transformer neural processes: Uncertainty-aware meta learning via sequence modeling},
author={Nguyen, Tung and Grover, Aditya},
journal={arXiv preprint arXiv:2207.04179},
year={2022}
}
Acknowledgement
The implementation of the baselines is borrowed from the official code base of Bootstrapping Neural Processes.