Regularized Evolution for Image Classifier Architecture Search

  • 2018-02-05 18:20:52
  • Esteban Real, Alok Aggarwal, Yanping Huang, Quoc V Le
  • 70

Abstract

The effort devoted to hand-crafting image classifiers has motivated the useof architecture search to discover them automatically. Reinforcement learningand evolution have both shown promise for this purpose. This study introduces aregularized version of a popular asynchronous evolutionary algorithm. Werigorously compare it to the non-regularized form and to a highly-successfulreinforcement learning baseline. Using the same hardware, compute effort andneural network training code, we conduct repeated experiments side-by-side,exploring different datasets, search spaces and scales. We show regularizedevolution consistently produces models with similar or higher accuracy, acrossa variety of contexts without need for re-tuning parameters. In addition,regularized evolution exhibits considerably better performance thanreinforcement learning at early search stages, suggesting it may be the betterchoice when fewer compute resources are available. This constitutes the firstcontrolled comparison of the two search algorithms in this context. Finally, wepresent new architectures discovered with regularized evolution that wenickname AmoebaNets. These models set a new state of the art for CIFAR-10 (meantest error = 2.13%) and mobile-size ImageNet (top-5 accuracy = 92.1% with 5.06Mparameters), and reach the current state of the art for ImageNet (top-5accuracy = 96.2%).

 

Introduction (beta)

None

 

Conclusion (beta)

None