Abstract
Deep neural networks have been found vulnerable to noises like adversarialexamples and corruption in practice. A number of adversarial defense methodshave been developed, which indeed improve the model robustness towardsadversarial examples in practice. However, only relying on training with thedata mixed with noises, most of them still fail to defend the generalized typesof noises. Motivated by the fact that hidden layers play a very important rolein maintaining a robust model, this paper comes up with a simple yet powerfultraining algorithm named Adversarial Noise Propagation (ANP) that injectsdiversified noises into the hidden layers in a layer-wise manner. We show thatANP can be efficiently implemented by exploiting the nature of the popularbackward-forward training style for deep models. To comprehensively understandthe behaviors and contributions of hidden layers, we further explore theinsights from hidden representation insensitivity and human vision perceptionalignment. Extensive experiments on MNIST, CIFAR-10, CIFAR-10-C, CIFAR-10-P andImageNet demonstrate that ANP enables the strong robustness for deep modelsagainst the generalized noises including both adversarial and corrupted ones,and significantly outperforms various adversarial defense methods.