NVAE: A Deep Hierarchical Variational Autoencoder

  • 2020-07-08 04:56:56
  • Arash Vahdat, Jan Kautz
  • 128

Abstract

Normalizing flows, autoregressive models, variational autoencoders (VAEs),and deep energy-based models are among competing likelihood-based frameworksfor deep generative learning. Among them, VAEs have the advantage of fast andtractable sampling and easy-to-access encoding networks. However, they arecurrently outperformed by other models such as normalizing flows andautoregressive models. While the majority of the research in VAEs is focused onthe statistical challenges, we explore the orthogonal direction of carefullydesigning neural architectures for hierarchical VAEs. We propose Nouveau VAE(NVAE), a deep hierarchical VAE built for image generation using depth-wiseseparable convolutions and batch normalization. NVAE is equipped with aresidual parameterization of Normal distributions and its training isstabilized by spectral regularization. We show that NVAE achievesstate-of-the-art results among non-autoregressive likelihood-based models onthe MNIST, CIFAR-10, and CelebA HQ datasets and it provides a strong baselineon FFHQ. For example, on CIFAR-10, NVAE pushes the state-of-the-art from 2.98to 2.91 bits per dimension, and it produces high-quality images on CelebA HQ asshown in Fig. 1. To the best of our knowledge, NVAE is the first successful VAEapplied to natural images as large as 256$\times$256 pixels.

 

Quick Read (beta)

loading the full paper ...