pursu / Awesome-NAS

A curated list of neural architecture search (NAS) resources.

Awesome NAS

A curated list of neural architecture search and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search.

Please feel free to pull requests or open an issue to add papers.

Table of Contents

• Neural Architecture Search
  • 2019 Venues
  • 2018 Venues
  • 2017 Venues
  • Previous Venues
  • Blogs
  • arXiv
• Benchmark on ImageNet

Neural Architecture Search (NAS)

Type	G	RL	EA	PD	Other
Explanation	gradient-based	reinforcement learning	evaluationary algorithm	performance prediction	other types

2019

Title	Venue	Type	Code
Neural architecture search: A survey	JMLR	Survey	-
DARTS: Differentiable Architecture Search	ICLR	G	github
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware	ICLR	RL/G	github
Graph HyperNetworks for Neural Architecture Search	ICLR	G	-
Learnable Embedding Space for Efficient Neural Architecture Compression	ICLR	Other	github
Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution	ICLR	EA	-
SNAS: stochastic neural architecture search	ICLR	G	-
Searching for A Robust Neural Architecture in Four GPU Hours	CVPR	G	github
ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation	CVPR	-	-
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search	CVPR	EA	github
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search	CVPR	G	-
RENAS: Reinforced Evolutionary Neural Architecture Search	CVPR	G	-
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation	CVPR	G	GitHub
MnasNet: Platform-Aware Neural Architecture Search for Mobile	CVPR	RL	Github
MFAS: Multimodal Fusion Architecture Search	CVPR	EA	-
A Neurobiological Evaluation Metric for Neural Network Model Search	CVPR	Other	-
Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells	CVPR	RL	-
Customizable Architecture Search for Semantic Segmentation	CVPR	-	-
Regularized Evolution for Image Classifier Architecture Search	AAAI	EA	-
AutoAugment: Learning Augmentation Policies from Data	CVPR	RL	-
Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules	ICML	EA	-
The Evolved Transformer	ICML	EA	Github
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks	ICML	RL	-
NAS-Bench-101: Towards Reproducible Neural Architecture Search	ICML	Other	Github

2018

Title	Venue	Type	Code
Towards Automatically-Tuned Deep Neural Networks	BOOK	-	GitHub
Efficient Architecture Search by Network Transformation	AAAI	RL	github
Learning Transferable Architectures for Scalable Image Recognition	CVPR	RL	github
N2N learning: Network to Network Compression via Policy Gradient Reinforcement Learning	ICLR	RL	-
Practical Block-wise Neural Network Architecture Generation	CVPR	RL	-
Path-Level Network Transformation for Efficient Architecture Search	ICML	RL	github
Hierarchical Representations for Efficient Architecture Search	ICLR	EA	-
Understanding and Simplifying One-Shot Architecture Search	ICML	G	-
SMASH: One-Shot Model Architecture Search through HyperNetworks	ICLR	G	github
Neural Architecture Optimization	NeurIPS	G	github
Searching for efficient multi-scale architectures for dense image prediction	NeurIPS	Other	-
Progressive Neural Architecture Search	ECCV	PD	github
Neural Architecture Search with Bayesian Optimisation and Optimal Transport	NeurIPS	Other	github
Differentiable Neural Network Architecture Search	ICLR-W	G	-
Accelerating Neural Architecture Search using Performance Prediction	ICLR-W	PD	-

2017

Title	Venue	Type	Code
Neural Architecture Search with Reinforcement Learning	ICLR	RL	-
Designing Neural Network Architectures using Reinforcement Learning	ICLR	RL	-
Neural Optimizer Search with Reinforcement Learning	ICML	RL	-
Learning Curve Prediction with Bayesian Neural Networks	ICLR	PD	-
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization	ICLR	PD	-
Hyperparameter Optimization: A Spectral Approach	NeurIPS-W	Other	github
Learning to Compose Domain-Specific Transformations for Data Augmentation	NeurIPS	-	-

2012-2016

Title	Venue	Type	Code
Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves	IJCAI	PD	github

Blogs

• What’s the deal with Neural Architecture Search?
• Google Could AutoML
• PocketFlow
• AutoML Challenge
• AutoDL Challenge

arXiv

• Network Pruning via Transformable Architecture Search [pdf]
  • Xuanyi Dong, Yi Yang
• Population Based Training of Neural Networks [pdf]
  • Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, Koray Kavukcuoglu. arXiv 1711
• NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search [pdf]
  • Lu, Zhichao and Whalen, Ian and Boddeti, Vishnu and Dhebar, Yashesh and Deb, Kalyanmoy and Goodman, Erik and Banzhaf, Wolfgang, arXiv 1810
• Random Search and Reproducibility for Neural Architecture Search
  • Liam Li, Ameet Talwalkar. arXiv 1901
• Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells [pdf]
  • Nekrasov, Vladimir and Chen, Hao and Shen, Chunhua and Reid, Ian. arXiv 1810
• Training Frankenstein’s Creature to Stack: HyperTree Architecture Search [pdf]
  • Andrew Hundt, Varun Jain, Chris Paxton, Gregory D. Hager. arXiv 1810
• Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search [pdf] [code]

Benchmark on ImageNet

Architecture	Top-1 (%)	Top-5 (%)	Params (M)	+x (M)	GPU	Days
Inception-v1	30.2	10.1	6.6	1448	-	-
MobileNet-v1	29.4	10.5	4.2	569	-	-
ShuffleNet	26.3	-	~5	524	-	-
NASNet-A	26.0	8.4	5.3	564	450	3-4
NASNet-B	27.2	8.7	5.3	488	450	3-4
NASNet-C	27.5	9.0	4.9	558	450	3-4
AmobebaNet-A	25.5	8.0	5.1	555	450	7
AmobebaNet-B	26.0	8.5	5.3	555	450	7
AmobebaNet-C	24.3	7.6	6.4	555	450	7
Progressive NAS	25.8	8.1	5.1	588	100	1.5
DARTS-V2	26.9	9.0	4.9	595	1	1
GDAS	26.0	8.5	5.3	581	1	0.21