Abstract
The development of diffusion-based generative models over the past decade haslargely proceeded independently of progress in representation learning. Thesediffusion models typically rely on regression-based objectives and generallylack explicit regularization. In this work, we propose \textit{DispersiveLoss}, a simple plug-and-play regularizer that effectively improvesdiffusion-based generative models. Our loss function encourages internalrepresentations to disperse in the hidden space, analogous to contrastiveself-supervised learning, with the key distinction that it requires no positivesample pairs and therefore does not interfere with the sampling process usedfor regression. Compared to the recent method of representation alignment(REPA), our approach is self-contained and minimalist, requiring nopre-training, no additional parameters, and no external data. We evaluateDispersive Loss on the ImageNet dataset across a range of models and reportconsistent improvements over widely used and strong baselines. We hope our workwill help bridge the gap between generative modeling and representationlearning.