Self Normalizing Flows

  • 2020-11-14 09:51:51
  • T. Anderson Keller, Jorn W. T. Peters, Priyank Jaini, Emiel Hoogeboom, Patrick Forré, Max Welling
  • 85

Abstract

Efficient gradient computation of the Jacobian determinant term is a coreproblem of the normalizing flow framework. Thus, most proposed flow modelseither restrict to a function class with easy evaluation of the Jacobiandeterminant, or an efficient estimator thereof. However, these restrictionslimit the performance of such density models, frequently requiring significantdepth to reach desired performance levels. In this work, we propose SelfNormalizing Flows, a flexible framework for training normalizing flows byreplacing expensive terms in the gradient by learned approximate inverses ateach layer. This reduces the computational complexity of each layer's exactupdate from $\mathcal{O}(D^3)$ to $\mathcal{O}(D^2)$, allowing for the trainingof flow architectures which were otherwise computationally infeasible, whilealso providing efficient sampling. We show experimentally that such models areremarkably stable and optimize to similar data likelihood values as their exactgradient counterparts, while surpassing the performance of their functionallyconstrained counterparts.

 

Quick Read (beta)

loading the full paper ...