Autoregressive Energy Machines

  • 2019-04-11 11:11:01
  • Charlie Nash, Conor Durkan
  • 108

Abstract

Neural density estimators are flexible families of parametric models whichhave seen widespread use in unsupervised machine learning in recent years.Maximum-likelihood training typically dictates that these models be constrainedto specify an explicit density. However, this limitation can be overcome byinstead using a neural network to specify an energy function, or unnormalizeddensity, which can subsequently be normalized to obtain a valid distribution.The challenge with this approach lies in accurately estimating the normalizingconstant of the high-dimensional energy function. We propose the AutoregressiveEnergy Machine, an energy-based model which simultaneously learns anunnormalized density and computes an importance-sampling estimate of thenormalizing constant for each conditional in an autoregressive decomposition.The Autoregressive Energy Machine achieves state-of-the-art performance on asuite of density-estimation tasks.

 

Quick Read (beta)

This feature is not avaialbe for this paper.