A curated list of neural architecture search and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search.
Please feel free to pull requests or open an issue to add papers.
- Neural Architecture Search with Reinforcement Learning [pdf]
- Barret Zoph and Quoc V. Le. ICLR 2017
- Designing Neural Network Architectures using Reinforcement Learning [pdf]
- Baker, Bowen and Gupta, Otkrist and Naik, Nikhil and Raskar, Ramesh. ICLR 2017
- Neural Optimizer Search with Reinforcement Learning [pdf]
- Bello, Irwan and Zoph, Barret and Vasudevan, Vijay and Le, Quoc V. ICML 2017
- Efficient Architecture Search by Network Transformation [pdf]
- Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, Jun Wang. AAAI 2018
- Learning Transferable Architectures for Scalable Image Recognition [pdf]
- Barret Zoph, Vijay Vasudevan, Jonathon Shlens, Quoc V. Le. CVPR 2018
- Practical Block-wise Neural Network Architecture Generation [pdf]
- Zhong, Zhao and Yan, Junjie and Wu, Wei and Shao, Jing and Liu, Cheng-Lin. CVPR 2018
- Efficient Neural Architecture Search via Parameter Sharing [pdf]
- Pham, Hieu and Guan, Melody Y and Zoph, Barret and Le, Quoc V and Dean, Jeff. ICML 2018
- MnasNet: Platform-Aware Neural Architecture Search for Mobile [pdf] [code]
- Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Quoc V. Le. arXiv 2018.07
- Path-Level Network Transformation for Efficient Architecture Search
- Cai, Han and Yang, Jiacheng and Zhang, Weinan and Han, Song and Yu, Yong. ICML 2018
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- Cai, Han and Zhu, Ligeng and Han, Song. ICLR 2019
- Population Based Training of Neural Networks [pdf]
- Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, Koray Kavukcuoglu. arXiv 1711
- Large-Scale Evolution of Image Classifiers [pdf]
- Real, Esteban and Moore, Sherry and Selle, Andrew and Saxena, Saurabh and Suematsu, Yutaka Leon and Tan, Jie and Le, Quoc and Kurakin, Alex. ICML 2017
- Hierarchical Representations for Efficient Architecture Search [pdf]
- Hanxiao Liu, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu. ICLR 2018
- Regularized Evolution for Image Classifier Architecture Search [pdf]
- Real, Esteban and Aggarwal, Alok and Huang, Yanping and Le, Quoc V. ICML 2018 Workshop
- Differentiable Neural Network Architecture Search [pdf]
- Richard Shin, Charles Packer, Dawn Song, ICLR 2018 Workshop
- Understanding and Simplifying One-Shot Architecture Search [pdf]
- Bender, Gabriel and Kindermans, Pieter-Jan and Zoph, Barret and Vasudevan, Vijay and Le, Quoc. ICML 2018
- SMASH: One-Shot Model Architecture Search through HyperNetworks [pdf]
- Brock, Andrew and Lim, Theodore and Ritchie, James M and Weston, Nick. ICLR 2018
- Neural Architecture Optimization [pdf] [code]
- Luo, Renqian and Tian, Fei and Qin, Tao and Liu, Tie-Yan. NIPS 2018
- DARTS: Differentiable Architecture Search [pdf] [code]
- Hanxiao Liu, Karen Simonyan, Yiming Yang. ICLR 2019
- Graph HyperNetworks for Neural Architecture Search [pdf]
- Chris Zhang, Mengye Ren, Raquel Urtasun. ICLR 2019
- SNAS: stochastic neural architecture search
- Sirui Xie, Hehui Zheng, Chunxiao Liu, Liang Lin. ICLR 2019
- Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves [pdf] [code]
- Tobias Domhan, Jost Tobias Springenberg, Frank Hutter. IJCAI 2015
- Learning Curve Prediction with Bayesian Neural Networks [pdf]
- Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter. ICLR 2017
- Progressive Neural Architecture Search [pdf] [code]
- Chenxi Liu, Barret Zoph, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy. ECCV 2018
- Accelerating Neural Architecture Search using Performance Prediction [pdf]
- Bowen Baker, Otkrist Gupta, Ramesh Raskar, Nikhil Naik. ICLR 2018 Workshop
- NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search [pdf]
- Lu, Zhichao and Whalen, Ian and Boddeti, Vishnu and Dhebar, Yashesh and Deb, Kalyanmoy and Goodman, Erik and Banzhaf, Wolfgang, arXiv 1810
- Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution
- Thomas Elsken, Jan Hendrik Metzen, Frank Hutter. ICLR 2019
- Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization [pdf]
- Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar. ICLR 2017
- Hyperparameter Optimization: A Spectral Approach [pdf] [code]
- Elad Hazan, Adam Klivans, Yang Yuan. NIPS 2017 Workshop
- Neural Architecture Search with Bayesian Optimisation and Optimal Transport [pdf]
- Kandasamy, Kirthevasan and Neiswanger, Willie and Schneider, Jeff and Poczos, Barnabas and Xing, Eric. NIPS 2018
- Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells [pdf]
- Nekrasov, Vladimir and Chen, Hao and Shen, Chunhua and Reid, Ian. arXiv 1810
- Training Frankenstein’s Creature to Stack: HyperTree Architecture Search [pdf]
- Andrew Hundt, Varun Jain, Chris Paxton, Gregory D. Hager. arXiv 1810
- Searching for efficient multi-scale architectures for dense image prediction [pdf]
- Chen, Liang-Chieh and Collins, Maxwell D and Zhu, Yukun and Papandreou, George and Zoph, Barret and Schroff, Florian and Adam, Hartwig and Shlens, Jonathon. NIPS 2018
- Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation [pdf]
- arXiv 1901
- Neural Architecture Search: A Survey
- Thomas Elsken, Jan Hendrik Metzen, Frank Hutter. arXiv 1808
- Taking Human out of Learning Applications: A Survey on Automated Machine Learning [pdf]
- Yao Quanming, Wang Mengshuo, Jair Escalante Hugo, Guyon Isabelle, Hu Yi-Qi, Li Yu-Feng, Tu Wei-Wei, Yang Qiang, Yu Yang. arXiv 1810
| Architecture | Top-1 (%) | Top-5 (%) | Params (M) | +x (M) | GPU | Days |
|---|---|---|---|---|---|---|
| Inception-v1 | 30.2 | 10.1 | 6.6 | 1448 | - | - |
| MobileNet-v1 | 29.4 | 10.5 | 4.2 | 569 | - | - |
| ShuffleNet | 26.3 | - | ~5 | 524 | - | - |
| NASNet-A | 26.0 | 8.4 | 5.3 | 564 | 450 | 3-4 |
| NASNet-B | 27.2 | 8.7 | 5.3 | 488 | 450 | 3-4 |
| NASNet-C | 27.5 | 9.0 | 4.9 | 558 | 450 | 3-4 |
| AmobebaNet-A | 25.5 | 8.0 | 5.1 | 555 | 450 | 7 |
| AmobebaNet-B | 26.0 | 8.5 | 5.3 | 555 | 450 | 7 |
| AmobebaNet-C | 24.3 | 7.6 | 6.4 | 555 | 450 | 7 |
| Progressive NAS | 25.8 | 8.1 | 5.1 | 588 | 100 | 1.5 |
| DARTS-V2 | 26.9 | 9.0 | 4.9 | 595 | 1 | 4 |