Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation as a Regularizer

  • 2018-06-13 19:32:59
  • Vikas Verma, Alex Lamb, Christopher Beckham, Aaron Courville, Ioannis Mitliagkis, Yoshua Bengio
  • 50

Abstract

Deep networks often perform well on the data manifold on which they aretrained, yet give incorrect (and often very confident) answers when evaluatedon points from off of the training distribution. This is exemplified by theadversarial examples phenomenon but can also be seen in terms of modelgeneralization and domain shift. We propose Manifold Mixup which encourages thenetwork to produce more reasonable and less confident predictions at pointswith combinations of attributes not seen in the training set. This isaccomplished by training on convex combinations of the hidden staterepresentations of data samples. Using this method, we demonstrate improvedsemi-supervised learning, learning with limited labeled data, and robustness toadversarial examples. Manifold Mixup requires no (significant) additionalcomputation. Analytical experiments on both real data and synthetic datadirectly support our hypothesis for why the Manifold Mixup method improvesresults.

 

Quick Read (beta)

loading the full paper ...