juliuskunze / eve

Eve optimizer from Adaptive Optimization with Examplewise Gradients

Adaptive Optimization with Examplewise Gradients

Implementation of the Eve optimizer and experiments from Adaptive Optimization with Examplewise Gradients.

If you find this code useful, please reference in your paper:

@article{kunze2021eve,
title={Adaptive Optimization with Examplewise Gradients},
author={Kunze, Julius and Townsend, Jamie and Barber, David},
journal={arXiv preprint arXiv:2112.00174},
year={2021}
}

Install

Install JAX.

Install dependencies for experiments of choice:

pip install -r <cifar10|seq2seq|wmt|ppo|pixelcnn>/requirements.txt

Run

See README.mds in subfolders to run experiments. Default configs are as used in the paper. Results are logged to wandb.