What Exactly Does Guidance Do in Masked Discrete Diffusion Models

  • 2025-06-12 18:59:19
  • He Ye, Rojas Kevin, Tao Molei
  • 0

Abstract

We study masked discrete diffusion models with classifier-free guidance(CFG). Assuming no score error nor discretization error, we derive an explicitsolution to the guided reverse dynamics, so that how guidance influences thesampling behavior can be precisely characterized. When the full datadistribution is a mixture over classes and the goal is to sample from aspecific class, guidance amplifies class-specific regions while suppressesregions shared with other classes. This effect depends on the guidance strength$w$ and induces distinct covariance structures in the sampled distribution.Notably, we observe quantitatively different behaviors in $1$D and $2$D. Wealso show that for large $w$, the decay rate of the total variation($\mathrm{TV}$) along the reverse dynamics is double-exponential in $w$ forboth $1$D and $2$D. These findings highlight the role of guidance, not just inshaping the output distribution, but also in controlling the dynamics of thesampling trajectory. Our theoretical analysis is supported by experiments thatillustrate the geometric effects of guidance and its impact on convergence.

 

Quick Read (beta)

loading the full paper ...