ScapeQin / awesome-NAS

A curated list of neural architecture search (NAS) resources.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Awesome NAS Awesome

A curated list of neural architecture search and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search.

Please feel free to pull requests or open an issue to add papers.

Table of Contents

NAS

Reinforcement Learning

  • Neural Architecture Search with Reinforcement Learning [pdf]
    • Barret Zoph and Quoc V. Le. ICLR 2017
  • Designing Neural Network Architectures using Reinforcement Learning [pdf]
    • Baker, Bowen and Gupta, Otkrist and Naik, Nikhil and Raskar, Ramesh. ICLR 2017
  • Neural Optimizer Search with Reinforcement Learning [pdf]
    • Bello, Irwan and Zoph, Barret and Vasudevan, Vijay and Le, Quoc V. ICML 2017
  • Efficient Architecture Search by Network Transformation [pdf]
    • Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, Jun Wang. AAAI 2018
  • Learning Transferable Architectures for Scalable Image Recognition [pdf]
    • Barret Zoph, Vijay Vasudevan, Jonathon Shlens, Quoc V. Le. CVPR 2018
  • Practical Block-wise Neural Network Architecture Generation [pdf]
    • Zhong, Zhao and Yan, Junjie and Wu, Wei and Shao, Jing and Liu, Cheng-Lin. CVPR 2018
  • Efficient Neural Architecture Search via Parameter Sharing [pdf]
    • Pham, Hieu and Guan, Melody Y and Zoph, Barret and Le, Quoc V and Dean, Jeff. ICML 2018
  • MnasNet: Platform-Aware Neural Architecture Search for Mobile [pdf] [code]
    • Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Quoc V. Le. arXiv 2018.07
  • Path-Level Network Transformation for Efficient Architecture Search
    • Cai, Han and Yang, Jiacheng and Zhang, Weinan and Han, Song and Yu, Yong. ICML 2018
  • ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
    • Cai, Han and Zhu, Ligeng and Han, Song. ICLR 2019

Evolutionary Algorithm

  • Population Based Training of Neural Networks [pdf]
    • Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, Koray Kavukcuoglu. arXiv 1711
  • Large-Scale Evolution of Image Classifiers [pdf]
    • Real, Esteban and Moore, Sherry and Selle, Andrew and Saxena, Saurabh and Suematsu, Yutaka Leon and Tan, Jie and Le, Quoc and Kurakin, Alex. ICML 2017
  • Hierarchical Representations for Efficient Architecture Search [pdf]
    • Hanxiao Liu, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu. ICLR 2018
  • Regularized Evolution for Image Classifier Architecture Search [pdf]
    • Real, Esteban and Aggarwal, Alok and Huang, Yanping and Le, Quoc V. ICML 2018 Workshop

Gradient-based Approach

  • Differentiable Neural Network Architecture Search [pdf]
    • Richard Shin, Charles Packer, Dawn Song, ICLR 2018 Workshop
  • Understanding and Simplifying One-Shot Architecture Search [pdf]
    • Bender, Gabriel and Kindermans, Pieter-Jan and Zoph, Barret and Vasudevan, Vijay and Le, Quoc. ICML 2018
  • SMASH: One-Shot Model Architecture Search through HyperNetworks [pdf]
    • Brock, Andrew and Lim, Theodore and Ritchie, James M and Weston, Nick. ICLR 2018
  • Neural Architecture Optimization [pdf] [code]
    • Luo, Renqian and Tian, Fei and Qin, Tao and Liu, Tie-Yan. NIPS 2018
  • DARTS: Differentiable Architecture Search [pdf] [code]
    • Hanxiao Liu, Karen Simonyan, Yiming Yang. ICLR 2019
  • Graph HyperNetworks for Neural Architecture Search [pdf]
    • Chris Zhang, Mengye Ren, Raquel Urtasun. ICLR 2019
  • SNAS: stochastic neural architecture search
    • Sirui Xie, Hehui Zheng, Chunxiao Liu, Liang Lin. ICLR 2019

Performance Prediction-based

  • Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves [pdf] [code]
    • Tobias Domhan, Jost Tobias Springenberg, Frank Hutter. IJCAI 2015
  • Learning Curve Prediction with Bayesian Neural Networks [pdf]
    • Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter. ICLR 2017
  • Progressive Neural Architecture Search [pdf] [code]
    • Chenxi Liu, Barret Zoph, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, Kevin Murphy. ECCV 2018
  • Accelerating Neural Architecture Search using Performance Prediction [pdf]
    • Bowen Baker, Otkrist Gupta, Ramesh Raskar, Nikhil Naik. ICLR 2018 Workshop

Multi-Objective NAS

  • NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search [pdf]
    • Lu, Zhichao and Whalen, Ian and Boddeti, Vishnu and Dhebar, Yashesh and Deb, Kalyanmoy and Goodman, Erik and Banzhaf, Wolfgang, arXiv 1810
  • Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution
    • Thomas Elsken, Jan Hendrik Metzen, Frank Hutter. ICLR 2019

Others

  • Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization [pdf]
    • Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar. ICLR 2017
  • Hyperparameter Optimization: A Spectral Approach [pdf] [code]
    • Elad Hazan, Adam Klivans, Yang Yuan. NIPS 2017 Workshop
  • Neural Architecture Search with Bayesian Optimisation and Optimal Transport [pdf]
    • Kandasamy, Kirthevasan and Neiswanger, Willie and Schneider, Jeff and Poczos, Barnabas and Xing, Eric. NIPS 2018

NAS for Application

  • Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells [pdf]
    • Nekrasov, Vladimir and Chen, Hao and Shen, Chunhua and Reid, Ian. arXiv 1810
  • Training Frankenstein’s Creature to Stack: HyperTree Architecture Search [pdf]
    • Andrew Hundt, Varun Jain, Chris Paxton, Gregory D. Hager. arXiv 1810
  • Searching for efficient multi-scale architectures for dense image prediction [pdf]
    • Chen, Liang-Chieh and Collins, Maxwell D and Zhu, Yukun and Papandreou, George and Zoph, Barret and Schroff, Florian and Adam, Hartwig and Shlens, Jonathon. NIPS 2018
  • Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation [pdf]
    • arXiv 1901

Survey

  • Neural Architecture Search: A Survey
    • Thomas Elsken, Jan Hendrik Metzen, Frank Hutter. arXiv 1808
  • Taking Human out of Learning Applications: A Survey on Automated Machine Learning [pdf]
    • Yao Quanming, Wang Mengshuo, Jair Escalante Hugo, Guyon Isabelle, Hu Yi-Qi, Li Yu-Feng, Tu Wei-Wei, Yang Qiang, Yu Yang. arXiv 1810

Benchmark on ImageNet

Architecture Top-1 (%) Top-5 (%) Params (M) +x (M) GPU Days
Inception-v1 30.2 10.1 6.6 1448 - -
MobileNet-v1 29.4 10.5 4.2 569 - -
ShuffleNet 26.3 - ~5 524 - -
NASNet-A 26.0 8.4 5.3 564 450 3-4
NASNet-B 27.2 8.7 5.3 488 450 3-4
NASNet-C 27.5 9.0 4.9 558 450 3-4
AmobebaNet-A 25.5 8.0 5.1 555 450 7
AmobebaNet-B 26.0 8.5 5.3 555 450 7
AmobebaNet-C 24.3 7.6 6.4 555 450 7
Progressive NAS 25.8 8.1 5.1 588 100 1.5
DARTS-V2 26.9 9.0 4.9 595 1 4

About

A curated list of neural architecture search (NAS) resources.

License:MIT License