Abstract
Autoencoders provide a powerful framework for learning compressedrepresentations by encoding all of the information needed to reconstruct a datapoint in a latent code. In some cases, autoencoders can "interpolate": Bydecoding the convex combination of the latent codes for two datapoints, theautoencoder can produce an output which semantically mixes characteristics fromthe datapoints. In this paper, we propose a regularization procedure whichencourages interpolated outputs to appear more realistic by fooling a criticnetwork which has been trained to recover the mixing coefficient frominterpolated data. We then develop a simple benchmark task where we canquantitatively measure the extent to which various autoencoders can interpolateand show that our regularizer dramatically improves interpolation in thissetting. We also demonstrate empirically that our regularizer produces latentcodes which are more effective on downstream tasks, suggesting a possible linkbetween interpolation abilities and learning useful representations.