Abstract
Scoring matching (SM), and its related counterpart, Stein discrepancy (SD)have achieved great success in model training and evaluations. However, recentresearch shows their limitations when dealing with certain types ofdistributions. One possible fix is incorporating the original score matching(or Stein discrepancy) with a diffusion matrix, which is called diffusion scorematching (DSM) (or diffusion Stein discrepancy (DSD)). However, the lack ofinterpretation of the diffusion limits its usage within simple distributionsand manually chosen matrix. In this work, we plan to fill this gap byinterpreting the diffusion matrix using normalizing flows. Specifically, wetheoretically prove that DSM (or DSD) is equivalent to the original scorematching (or Stein discrepancy) evaluated in the transformed space defined bythe normalizing flow, where the diffusion matrix is the inverse of the flow'sJacobian matrix. In addition, we also build its connection to Riemannianmanifolds and further extend it to continuous flows, where the change of DSM ischaracterized by an ODE.