Abstract
We introduce Self Forcing, a novel training paradigm for autoregressive videodiffusion models. It addresses the longstanding issue of exposure bias, wheremodels trained on ground-truth context must generate sequences conditioned ontheir own imperfect outputs during inference. Unlike prior methods that denoisefuture frames based on ground-truth context frames, Self Forcing conditionseach frame's generation on previously self-generated outputs by performingautoregressive rollout with key-value (KV) caching during training. Thisstrategy enables supervision through a holistic loss at the video level thatdirectly evaluates the quality of the entire generated sequence, rather thanrelying solely on traditional frame-wise objectives. To ensure trainingefficiency, we employ a few-step diffusion model along with a stochasticgradient truncation strategy, effectively balancing computational cost andperformance. We further introduce a rolling KV cache mechanism that enablesefficient autoregressive video extrapolation. Extensive experiments demonstratethat our approach achieves real-time streaming video generation with sub-secondlatency on a single GPU, while matching or even surpassing the generationquality of significantly slower and non-causal diffusion models. Projectwebsite: http://self-forcing.github.io/