Abstract
We demonstrate, theoretically and empirically, that adversarial robustnesscan significantly benefit from semisupervised learning. Theoretically, werevisit the simple Gaussian model of Schmidt et al. that shows a samplecomplexity gap between standard and robust classification. We prove thatunlabeled data bridges this gap: a simple semisupervised learning procedure(self-training) achieves high robust accuracy using the same number of labelsrequired for achieving high standard accuracy. Empirically, we augment CIFAR-10with 500K unlabeled images sourced from 80 Million Tiny Images and use robustself-training to outperform state-of-the-art robust accuracies by over 5 pointsin (i) $\ell_\infty$ robustness against several strong attacks via adversarialtraining and (ii) certified $\ell_2$ and $\ell_\infty$ robustness viarandomized smoothing. On SVHN, adding the dataset's own extra training set withthe labels removed provides gains of 4 to 10 points, within 1 point of the gainfrom using the extra labels.