Architecture Search

Labels

• Optimization Method
  • RL–> Reinforcement Learning
  • EA–> Evolution Algorithm
  • GD–> Gradient Descent
  • BO–> Bayesian Optimisation
  • MCTS –> Monte Carlo Tree Search
  • SMBO –> Sequential Model-Based Optimization
  • 1S –> 1-Shot Learning
• Objective Function
  • DE –> DEvice-related: inference time, memory usage, power consumption
• Training
  • FT –> FineTune on pretrained models
  • SR –> ScRatch
  • TL –> Transfer Learning between tasks
  • PS –> Parameter Sharing
  • NM –> Network Morphisms
  • KT –> Knowledge Transfer
• Search Level
  • TP –> ToPology of connection paths
  • SG –> SubGraph within a large computational graph
  • SM –> Frequent Computational Subgraph Mining
  • RNN
  • SE –> Shink and Expand
  • BC –> Block-wise Component
  • ML –> Modeling Language
  • MM –> Modularized Morphing
• Accurarcy Computation
  • PP –> Performance Prediction
  • ST –> STatistics derived from filter feature maps
  • WP –> Weight Prediction

2018

Neural Architecture Optimization
[arXiv:1808.07233]

Designing Adaptive Neural Networks for Energy-Constrained Image Classification
[arXiv:1808.01550]

Reinforced Evolutionary Neural Architecture Search
[arXiv:1808.00193]

MnasNet: Platform-Aware Neural Architecture Search for Mobile
[arXiv:1807.11626]

MaskConnect: Connectivity Learning by Gradient Descent
[arXiv:1807.11473] [ECCV’18]

Efficient Neural Architecture Search with Network Morphism
[arXiv:1806.10282] [code]

DARTS: Differentiable Architecture Search :star:
[arXiv:1806.09055] [code] –> GD

DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures
[arXiv:1806.08198] [ICLR’18 Workshop] –> DE

Path-Level Network Transformation for Efficient Architecture Search
[arXiv:1806.02639] [code] [ICML’18] –> FT TP RL

AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
[arXiv:1805.07440] –> SR MCTS PP

Neural Architecture Construction using EnvelopeNets
[arXiv:1803.06744] –> ST

Transfer Automatic Machine Learning
[arXiv:1803.02780] –> RL TL

Neural Architecture Search with Bayesian Optimisation and Optimal Transport
[arXiv:1802.07191] –> BO

Efficient Neural Architecture Search via Parameter Sharing :star:
[arXiv:1802.03268] [code] [ICML’18] –> PS RL SG

Regularized Evolution for Image Classifier Architecture Search
[arXiv:1802.01548] –> EA

GitGraph - from Computational Subgraphs to Smaller Architecture Search Spaces
[arXiv:1801.05159] [ICLR’18 Workshop] –> SM

A Flexible Approach to Automated RNN Architecture Generation
[arXiv:1712.07316] [ICLR’18 Workshop] –> RNN

Peephole: Predicting Network Performance Before Training :star:
[arXiv:1712.03351] –> PP

Progressive Neural Architecture Search
[arXiv:1712.00559] –> SMBO

MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks :star:
[arXiv:1711.06798] –> SE

Simple And Efficient Architecture Search for Convolutional Neural Networks :star:
[arXiv:1711.04528] [ICLR’18 Workshop] –> NM

Hierarchical Representations for Efficient Architecture Search
[arXiv:1711.00436] [ICLR’18] –> EA TP

Practical Block-wise Neural Network Architecture Generation
[arXiv:1708.05552] [CVPR’18] –> BC

SMASH: One-Shot Model Architecture Search through HyperNetworks
[arXiv:1708.05344] [code] [ICLR’18] –> WP

Learning Transferable Architectures for Scalable Image Recognition
[arXiv:1707.07012] –> BC

Efficient Architecture Search by Network Transformation
[arXiv:1707.04873] [code] [AAAI’18] –> FT RL SE

Learning Time/Memory-Efficient Deep Architectures with Budgeted Super Networks :star:
[arXiv:1706.00046] [code] [CVPR’18] –> DE GD

Accelerating Neural Architecture Search using Performance Prediction
[arXiv:1705.10823] [ICLR’18 Workshop] –> PP

Understanding and Simplifying One-Shot Architecture Search :star:
[ICML’18] –> 1S

2017

DeepArchitect: Automatically Designing and Training Deep Architectures
[arXiv:1704.08792] [code] –> ML

Genetic CNN
[arXiv:1703.01513] [code] [ICCV’17] –> EA

Modularized Morphing of Neural Networks
[arXiv:1701.03281] [ICLR’17 Workshop] –> FT MM

Large-Scale Evolution of Image Classifiers
[arXiv:1703.01041] [ICML’17] –> EA

Designing Neural Network Architectures using Reinforcement Learning
[arXiv:1611.02167] [code] [ICLR’17] –> RL

Learning Curve Prediction with Bayesian Neural Networks
[ICLR’17] –> PP

Neural Architecture Search with Reinforcement Learning
[arXiv:1611.01578] [code (3rd)] [ICLR’17] –> RL

2016

Convolutional Neural Fabrics
[arXiv:1606.02492] [code:Caffe] [code:PyTorch] [NIPS’16]

Network Morphism :star:
[arXiv:1603.01670] [ICML’16] –> FT MM

Net2Net: Accelerating Learning via Knowledge Transfer :star:
[arXiv:1511.05641] [ICLR’16] –> KT

~ 2015

A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks
[Artificial Life journal’09] [code] –> EA TP

Useful Link

1. https://www.ml4aad.org/automl/literature-on-neural-architecture-search/