Abstract
Deep networks often perform well on the data manifold on which they aretrained, yet give incorrect (and often very confident) answers when evaluatedon points from off of the training distribution. This is exemplified by theadversarial examples phenomenon but can also be seen in terms of modelgeneralization and domain shift. We propose Manifold Mixup which encourages thenetwork to produce more reasonable and less confident predictions at pointswith combinations of attributes not seen in the training set. This isaccomplished by training on convex combinations of the hidden staterepresentations of data samples. Using this method, we demonstrate improvedsemi-supervised learning, learning with limited labeled data, and robustness toadversarial examples. Manifold Mixup requires no (significant) additionalcomputation. Analytical experiments on both real data and synthetic datadirectly support our hypothesis for why the Manifold Mixup method improvesresults.