Abstract
Denoising Score Matching estimates the score of a noised version of a targetdistribution by minimizing a regression loss and is widely used to train thepopular class of Denoising Diffusion Models. A well known limitation ofDenoising Score Matching, however, is that it yields poor estimates of thescore at low noise levels. This issue is particularly unfavourable for problemsin the physical sciences and for Monte Carlo sampling tasks for which the scoreof the clean original target is known. Intuitively, estimating the score of aslightly noised version of the target should be a simple task in such cases. Inthis paper, we address this shortcoming and show that it is indeed possible toleverage knowledge of the target score. We present a Target Score Identity andcorresponding Target Score Matching regression loss which allows us to obtainscore estimates admitting favourable properties at low noise levels.