Abstract
We propose Efficient Neural Architecture Search (ENAS), a fast andinexpensive approach for automatic model design. In ENAS, a controller learnsto discover neural network architectures by searching for an optimal subgraphwithin a large computational graph. The controller is trained with policygradient to select a subgraph that maximizes the expected reward on thevalidation set. Meanwhile the model corresponding to the selected subgraph istrained to minimize a canonical cross entropy loss. Thanks to parameter sharingbetween child models, ENAS is fast: it delivers strong empirical performancesusing much fewer GPU-hours than all existing automatic model design approaches,and notably, 1000x less expensive than standard Neural Architecture Search. Onthe Penn Treebank dataset, ENAS discovers a novel architecture that achieves atest perplexity of 55.8, establishing a new state-of-the-art among all methodswithout post-training processing. On the CIFAR-10 dataset, ENAS designs novelarchitectures that achieve a test error of 2.89%, which is on par with NASNet(Zoph et al., 2018), whose test error is 2.65%.