Bidirectional Learning for Robust Neural Networks

  • 2018-12-05 16:15:45
  • Sidney Pontes-Filho, Marcus Liwicki
  • 0

Abstract

A multilayer perceptron can behave as a generative classifier by applyingbidirectional learning (BL). It consists of training an undirected neuralnetwork to map input to output and vice-versa; therefore it can produce aclassifier in one direction, and a generator in the opposite direction for thesame data. The learning process of BL tries to reproduce the neuroplasticitystated in Hebbian theory using only backward propagation of errors. In thispaper, two novel learning techniques are introduced which use BL for improvingrobustness to white noise static and adversarial examples. The first method isbidirectional propagation of errors, which the error propagation occurs inbackward and forward directions. Motivated by the fact that its generativemodel receives as input a constant vector per class, we introduce as a secondmethod the hybrid adversarial networks (HAN). Its generative model receives arandom vector as input and its training is based on generative adversarialnetworks (GAN). To assess the performance of BL, we perform experiments usingseveral architectures with fully and convolutional layers, with and withoutbias. Experimental results show that both methods improve robustness to whitenoise static and adversarial examples, and even increase accuracy, but havedifferent behavior depending on the architecture and task, being morebeneficial to use the one or the other. Nevertheless, HAN using a convolutionalarchitecture with batch normalization presents outstanding robustness, reachingstate-of-the-art accuracy on adversarial examples of hand-written digits.

 

Introduction (beta)

None

 

Conclusion (beta)

None