tinhb92 / Awesome-NAS

A curated list of neural architecture search (NAS) resources.

Awesome NAS

A curated list of neural architecture search and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search.

Please feel free to pull requests or open an issue to add papers.

Table of Contents

• Neural Architecture Search
  • 2019 Venues
  • 2018 Venues
  • 2017 Venues
  • Previous Venues
  • Blogs
  • arXiv
• NAS for Application
• Survey
• Benchmark on ImageNet

Neural Architecture Search (NAS)

Type	G	RL	EA	PD	Other
Explanation	gradient-based	reinforcement learning	evaluationary algorithm	performance prediction	other types

2019

Title	Venue	Type	Code
DARTS: Differentiable Architecture Search	ICLR	G	github
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware	ICLR	RL/G	github
Graph HyperNetworks for Neural Architecture Search	ICLR	G	-
Searching for A Robust Neural Architecture in Four GPU Hours	CVPR	G	github
Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution	ICLR	EA	-
Regularized Evolution for Image Classifier Architecture Search	AAAI	EA	-
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search	CVPR	EA	github
SNAS: stochastic neural architecture search	ICLR	G	-
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search	CVPR	G	-
Reinforced Evolutionary Neural Architecture Search	CVPR	G	-

2018

Title	Venue	Type	Code
Efficient Architecture Search by Network Transformation	AAAI	RL	github
Learning Transferable Architectures for Scalable Image Recognition	CVPR	RL	github
Practical Block-wise Neural Network Architecture Generation	CVPR	RL	-
Path-Level Network Transformation for Efficient Architecture Search	ICML	RL	github
Hierarchical Representations for Efficient Architecture Search	ICLR	EA	-
Understanding and Simplifying One-Shot Architecture Search	ICML	G	-
SMASH: One-Shot Model Architecture Search through HyperNetworks	ICLR	G	github
Neural Architecture Optimization	NeurIPS	G	github
Progressive Neural Architecture Search	ECCV	PD	github
Neural Architecture Search with Bayesian Optimisation and Optimal Transport	NeurIPS	Other	github
Differentiable Neural Network Architecture Search	ICLR-W	G	-
Accelerating Neural Architecture Search using Performance Prediction	ICLR-W	PD	-

2017

Title	Venue	Type	Code
Neural Architecture Search with Reinforcement Learning	ICLR	RL	-
Designing Neural Network Architectures using Reinforcement Learning	ICLR	RL	-
Neural Optimizer Search with Reinforcement Learning	ICML	RL	-
Learning Curve Prediction with Bayesian Neural Networks	ICLR	PD	-
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization	ICLR	PD	-
Hyperparameter Optimization: A Spectral Approach	NeurIPS-W	Other	github

2012-2016

Title	Venue	Type	Code
Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves	IJCAI	PD	github

Blogs

• What’s the deal with Neural Architecture Search?
• Google Could AutoML
• PocketFlow
• AutoML Challenge
• AutoDL Challenge

arXiv

• MnasNet: Platform-Aware Neural Architecture Search for Mobile [pdf] [code]
  • Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Quoc V. Le. arXiv 2018.07
• Population Based Training of Neural Networks [pdf]
  • Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, Koray Kavukcuoglu. arXiv 1711
• NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search [pdf]
  • Lu, Zhichao and Whalen, Ian and Boddeti, Vishnu and Dhebar, Yashesh and Deb, Kalyanmoy and Goodman, Erik and Banzhaf, Wolfgang, arXiv 1810
• Random Search and Reproducibility for Neural Architecture Search
  • Liam Li, Ameet Talwalkar. arXiv 1901

NAS for Application

• Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells [pdf]
  • Nekrasov, Vladimir and Chen, Hao and Shen, Chunhua and Reid, Ian. arXiv 1810
• Training Frankenstein’s Creature to Stack: HyperTree Architecture Search [pdf]
  • Andrew Hundt, Varun Jain, Chris Paxton, Gregory D. Hager. arXiv 1810
• Searching for efficient multi-scale architectures for dense image prediction [pdf]
  • Chen, Liang-Chieh and Collins, Maxwell D and Zhu, Yukun and Papandreou, George and Zoph, Barret and Schroff, Florian and Adam, Hartwig and Shlens, Jonathon. NeurIPS 2018
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation [pdf]
  • arXiv 1901
• Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search [pdf] [code]

Survey

• Neural Architecture Search: A Survey
  • Thomas Elsken, Jan Hendrik Metzen, Frank Hutter. arXiv 1808
• Taking Human out of Learning Applications: A Survey on Automated Machine Learning [pdf]
  • Yao Quanming, Wang Mengshuo, Jair Escalante Hugo, Guyon Isabelle, Hu Yi-Qi, Li Yu-Feng, Tu Wei-Wei, Yang Qiang, Yu Yang. arXiv 1810

Benchmark on ImageNet

Architecture	Top-1 (%)	Top-5 (%)	Params (M)	+x (M)	GPU	Days
Inception-v1	30.2	10.1	6.6	1448	-	-
MobileNet-v1	29.4	10.5	4.2	569	-	-
ShuffleNet	26.3	-	~5	524	-	-
NASNet-A	26.0	8.4	5.3	564	450	3-4
NASNet-B	27.2	8.7	5.3	488	450	3-4
NASNet-C	27.5	9.0	4.9	558	450	3-4
AmobebaNet-A	25.5	8.0	5.1	555	450	7
AmobebaNet-B	26.0	8.5	5.3	555	450	7
AmobebaNet-C	24.3	7.6	6.4	555	450	7
Progressive NAS	25.8	8.1	5.1	588	100	1.5
DARTS-V2	26.9	9.0	4.9	595	1	1
GDAS	26.0	8.5	5.3	581	1	0.21