hadisinaee / automl-resources

A list of resources, papers, and articles on AutoML, Automatic Hyperparameter Tuning, and Neural Architecture Search.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

This is a draft, and will become more curated and more organized as I get more involved with the topic. However, I think the listed resources are definitely worth going through.

Books

Essential Papers (General)

  • Taking Human out of Learning Applications: A Survey on Automated Machine Learning (paper) (2019)
  • Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search (paper) (2018)
  • Efficient and Robust Automated Machine Learning (paper) (NIPS 2015)

Hyperparameter Optimization

Essentials

  • Google Vizier: A Service for Black-Box Optimization (paper) (KDD 2017)

Advanced

  • Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization (paper) (2018)
  • Massively Parallel Hyperparameter Tuning (paper) (blog post) (2019)
  • Practical Hyperparameter Optimization for Deep Learning (paper) (ICLR 2018)

Bayesian Approaches

  • Martin Krasser's Guide to Bayesian Methods for Machine Learning (intro blog post) (Tutorials and Notebooks)
  • A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning (paper) (2010)
  • Taking the Human Out of the Loop: A Review of Bayesian Optimization (paper) (2016)

Architecture Search

Essentials

  • Neural Architecture Search with Reinforcement Learning (paper)
  • Progressive Neural Architecture Search (paper) (2018)

Advanced

  • Progressive Dynamic Hurdling - The Evolved Transformer (paper) (my slides) (2019)
  • Designing Neural Network Architectures using Reinforcement Learning (paper) (2017)
  • Genetic CNN (paper) (2017)
  • Large-Scale Evolution of Image Classifiers (paper) (2017)
  • AmoebaNet: Regularized Evolution for Image Classifier Architecture Search (paper) (2019)
  • DARTS: Differentiable Architecture Search (paper) (2018)
  • Learning to Learn Using Gradient Descent (paper) (ICANN 2001)
  • Learning to Learn without Gradient Descent by Gradient Descent (paper) (ICML 2017)

Other Related Literature

  • A Comparative Analysis of Selection Schemes Used in Genetic Algorithms (paper)

About

A list of resources, papers, and articles on AutoML, Automatic Hyperparameter Tuning, and Neural Architecture Search.