Embedded-model flows: Combining the inductive biases of model-free deep learning and explicit probabilistic modeling

  • 2021-10-12 14:12:16
  • Gianluigi Silvestri, Emily Fertig, Dave Moore, Luca Ambrogioni
  • 41

Abstract

Normalizing flows have shown great success as general-purpose densityestimators. However, many real world applications require the use ofdomain-specific knowledge, which normalizing flows cannot readily incorporate.We propose embedded-model flows(EMF), which alternate general-purposetransformations with structured layers that embed domain-specific inductivebiases. These layers are automatically constructed by converting user-specifieddifferentiable probabilistic models into equivalent bijective transformations.We also introduce gated structured layers, which allow bypassing the parts ofthe models that fail to capture the statistics of the data. We demonstrate thatEMFs can be used to induce desirable properties such as multimodality,hierarchical coupling and continuity. Furthermore, we show that EMFs enable ahigh performance form of variational inference where the structure of the priormodel is embedded in the variational architecture. In our experiments, we showthat this approach outperforms state-of-the-art methods in common structuredinference problems.

 

Quick Read (beta)

loading the full paper ...