Abstract
Adversarial examples are commonly viewed as a threat to ConvNets. Here wepresent an opposite perspective: adversarial examples can be used to improveimage recognition models if harnessed in the right manner. We propose AdvProp,an enhanced adversarial training scheme which treats adversarial examples asadditional examples, to prevent overfitting. Key to our method is the usage ofa separate auxiliary batch norm for adversarial examples, as they havedifferent underlying distributions to normal examples. We show that AdvProp improves a wide range of models on various imagerecognition tasks and performs better when the models are bigger. For instance,by applying AdvProp to the latest EfficientNet-B7 [28] on ImageNet, we achievesignificant improvements on ImageNet (+0.7%), ImageNet-C (+6.5%), ImageNet-A(+7.0%), Stylized-ImageNet (+4.8%). With an enhanced EfficientNet-B8, ourmethod achieves the state-of-the-art 85.5% ImageNet top-1 accuracy withoutextra data. This result even surpasses the best model in [20] which is trainedwith 3.5B Instagram images (~3000X more than ImageNet) and ~9.4X moreparameters. Models are available athttps://github.com/tensorflow/tpu/tree/master/models/official/efficientnet.