EfficientNetV2: Smaller Models and Faster Training

  • 2021-04-01 07:08:36
  • Mingxing Tan, Quoc V. Le
  • 79

Abstract

This paper introduces EfficientNetV2, a new family of convolutional networksthat have faster training speed and better parameter efficiency than previousmodels. To develop this family of models, we use a combination oftraining-aware neural architecture search and scaling, to jointly optimizetraining speed and parameter efficiency. The models were searched from thesearch space enriched with new ops such as Fused-MBConv. Our experiments showthat EfficientNetV2 models train much faster than state-of-the-art models whilebeing up to 6.8x smaller. Our training can be further sped up by progressively increasing the imagesize during training, but it often causes a drop in accuracy. To compensate forthis accuracy drop, we propose to adaptively adjust regularization (e.g.,dropout and data augmentation) as well, such that we can achieve both fasttraining and good accuracy. With progressive learning, our EfficientNetV2 significantly outperformsprevious models on ImageNet and CIFAR/Cars/Flowers datasets. By pretraining onthe same ImageNet21k, our EfficientNetV2 achieves 87.3% top-1 accuracy onImageNet ILSVRC2012, outperforming the recent ViT by 2.0% accuracy whiletraining 5x-11x faster using the same computing resources. Code will beavailable at https://github.com/google/automl/efficientnetv2.

 

Quick Read (beta)

loading the full paper ...