Interpreting diffusion score matching using normalizing flow

  • 2021-07-21 13:27:32
  • Wenbo Gong, Yingzhen Li
  • 14


Scoring matching (SM), and its related counterpart, Stein discrepancy (SD)have achieved great success in model training and evaluations. However, recentresearch shows their limitations when dealing with certain types ofdistributions. One possible fix is incorporating the original score matching(or Stein discrepancy) with a diffusion matrix, which is called diffusion scorematching (DSM) (or diffusion Stein discrepancy (DSD)). However, the lack ofinterpretation of the diffusion limits its usage within simple distributionsand manually chosen matrix. In this work, we plan to fill this gap byinterpreting the diffusion matrix using normalizing flows. Specifically, wetheoretically prove that DSM (or DSD) is equivalent to the original scorematching (or Stein discrepancy) evaluated in the transformed space defined bythe normalizing flow, where the diffusion matrix is the inverse of the flow'sJacobian matrix. In addition, we also build its connection to Riemannianmanifolds and further extend it to continuous flows, where the change of DSM ischaracterized by an ODE.


Quick Read (beta)

loading the full paper ...