Abstract
Although deep networks achieve strong accuracy on a range of computer visionbenchmarks, they remain vulnerable to adversarial attacks, where imperceptibleinput perturbations fool the network. We present both theoretical and empiricalanalyses that connect the adversarial robustness of a model to the number oftasks that it is trained on. Experiments on two datasets show that attackdifficulty increases as the number of target tasks increase. Moreover, ourresults suggest that when models are trained on multiple tasks at once, theybecome more robust to adversarial attacks on individual tasks. Whileadversarial defense remains an open challenge, our results suggest that deepnetworks are vulnerable partly because they are trained on too few tasks.