Abstract
We present a novel theoretical framework for understanding the expressivepower of coupling-based normalizing flows such as RealNVP. Despite theirprevalence in scientific applications, a comprehensive understanding ofcoupling flows remains elusive due to their restricted architectures. Existingtheorems fall short as they require the use of arbitrarily ill-conditionedneural networks, limiting practical applicability. Additionally, we demonstratethat these constructions inherently lead to volume-preserving flows, a propertywhich we show to be a fundamental constraint for expressivity. We propose a newdistributional universality theorem for coupling-based normalizing flows, whichovercomes several limitations of prior work. Our results support the generalwisdom that the coupling architecture is expressive and provide a nuanced viewfor choosing the expressivity of coupling functions, bridging a gap betweenempirical results and theoretical understanding.