Abstract
We present Pyramid Attention Broadcast (PAB), a real-time, high quality andtraining-free approach for DiT-based video generation. Our method is founded onthe observation that attention difference in the diffusion process exhibits aU-shaped pattern, indicating significant redundancy. We mitigate this bybroadcasting attention outputs to subsequent steps in a pyramid style. Itapplies different broadcast strategies to each attention based on theirvariance for best efficiency. We further introduce broadcast sequence parallelfor more efficient distributed inference. PAB demonstrates superior resultsacross three models compared to baselines, achieving real-time generation forup to 720p videos. We anticipate that our simple yet effective method willserve as a robust baseline and facilitate future research and application forvideo generation.