On the Universality of Coupling-based Normalizing Flows

  • 2024-02-09 17:51:43
  • Felix Draxler, Stefan Wahl, Christoph Schnörr, Ullrich Köthe
  • 0

Abstract

We present a novel theoretical framework for understanding the expressivepower of coupling-based normalizing flows such as RealNVP. Despite theirprevalence in scientific applications, a comprehensive understanding ofcoupling flows remains elusive due to their restricted architectures. Existingtheorems fall short as they require the use of arbitrarily ill-conditionedneural networks, limiting practical applicability. Additionally, we demonstratethat these constructions inherently lead to volume-preserving flows, a propertywhich we show to be a fundamental constraint for expressivity. We propose a newdistributional universality theorem for coupling-based normalizing flows, whichovercomes several limitations of prior work. Our results support the generalwisdom that the coupling architecture is expressive and provide a nuanced viewfor choosing the expressivity of coupling functions, bridging a gap betweenempirical results and theoretical understanding.

 

Quick Read (beta)

loading the full paper ...