Abstract
With the introduction of the variational autoencoder (VAE), probabilisticlatent variable models have received renewed attention as powerful generativemodels. However, their performance in terms of test likelihood and quality ofgenerated samples has been surpassed by autoregressive models withoutstochastic units. Furthermore, flow-based models have recently been shown to bean attractive alternative that scales well to high-dimensional data. In thispaper we close the performance gap by constructing VAE models that caneffectively utilize a deep hierarchy of stochastic variables and model complexcovariance structures. We introduce the Bidirectional-Inference VariationalAutoencoder (BIVA), characterized by a skip-connected generative model and aninference network formed by a bidirectional stochastic inference path. We showthat BIVA reaches state-of-the-art test likelihoods, generates sharp andcoherent natural images, and uses the hierarchy of latent variables to capturedifferent aspects of the data distribution. We observe that BIVA, in contrastto recent results, can be used for anomaly detection. We attribute this to thehierarchy of latent variables which is able to extract high-level semanticfeatures. Finally, we extend BIVA to semi-supervised classification tasks andshow that it performs comparably to state-of-the-art results by generativeadversarial networks.