Inductive Moment Matching

  • 2025-03-10 18:37:39
  • Linqi Zhou, Stefano Ermon, Jiaming Song
  • 0

Abstract

Diffusion models and Flow Matching generate high-quality samples but are slowat inference, and distilling them into few-step models often leads toinstability and extensive tuning. To resolve these trade-offs, we proposeInductive Moment Matching (IMM), a new class of generative models for one- orfew-step sampling with a single-stage training procedure. Unlike distillation,IMM does not require pre-training initialization and optimization of twonetworks; and unlike Consistency Models, IMM guarantees distribution-levelconvergence and remains stable under various hyperparameters and standard modelarchitectures. IMM surpasses diffusion models on ImageNet-256x256 with 1.99 FIDusing only 8 inference steps and achieves state-of-the-art 2-step FID of 1.98on CIFAR-10 for a model trained from scratch.

 

Quick Read (beta)

loading the full paper ...