Abstract
We present a method for generating video sequences with coherent motionbetween a pair of input key frames. We adapt a pretrained large-scaleimage-to-video diffusion model (originally trained to generate videos movingforward in time from a single input image) for key frame interpolation, i.e.,to produce a video in between two input frames. We accomplish this adaptationthrough a lightweight fine-tuning technique that produces a version of themodel that instead predicts videos moving backwards in time from a single inputimage. This model (along with the original forward-moving model) issubsequently used in a dual-directional diffusion sampling process thatcombines the overlapping model estimates starting from each of the twokeyframes. Our experiments show that our method outperforms both existingdiffusion-based methods and traditional frame interpolation techniques.