TPD: Temporal Prior Decoupling for Text-to-Video Diffusion Models

  • 2026-11-17 18:32:39
  • Taewon Kang, Matthias Zwicker
  • 0

Abstract

Text-to-video diffusion models generate temporally coherent content from natural language, yet when a prompt describes an early scene that persists while a new event emerges on top of it (such as "a tall sandcastle standing on a beach where a wave rushes in and washes it away"), generation frequently fails to realize the late-segment event in the corresponding frames. We identify this failure as Temporal Prior Suppression (TPS), in which the dominant prior of the early segment captures the cross-attention trajectory across the temporal axis and suppresses the guidance signal needed for late-segment realization. We introduce Temporal Prior Decoupling (TPD), a training-free heuristic that strengthens the suppressed late-segment signal during diffusion sampling. TPD constructs a temporal counterfactual by conditioning on the early segment alone, and defines the discrepancy between the full-prompt and counterfactual predictions as a suppressed signal direction. Rather than removing a direction as in prior subtractive projection methods, TPD adds this direction back to the guidance update whenever the alignment of the update with it falls below a threshold. The threshold follows a hand-designed schedule over diffusion timestep and video frame, so the correction acts mainly on late frames and structure-forming steps and leaves early frames largely untouched. TPD runs entirely within standard diffusion sampling without retraining and is defined purely in classifier-free guidance space, so it applies to any backbone that exposes a CFG update. Experiments show that TPD substantially improves late-concept realization while preserving temporal coherence and visual fidelity, and that the targeted suppression recurs across distinct text-to-video backbones.

 

Quick Read (beta)

loading the full paper ...