Adashift
Reproducing of the AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods as a part of ICLR Reproducibility Challange 2019. See our report.
Experiments
Synthetic Experiment
Logistic Regression on MNIST
W-GAN
NMT
Dependencies
- Python 3.4+,
- NumPy, Pandas
- PyTorch 0.4+
- SciPy, Matplotlib
- OpenNMT-py for NMT
- GPU for DL experiments
References
On the Convergence of Adam and Beyond
AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods