Abstract
We present an approach to modeling an image-space prior on scene motion. Ourprior is learned from a collection of motion trajectories extracted from realvideo sequences depicting natural, oscillatory dynamics such as trees, flowers,candles, and clothes swaying in the wind. We model this dense, long-term motionprior in the Fourier domain:given a single image, our trained model uses afrequency-coordinated diffusion sampling process to predict a spectral volume,which can be converted into a motion texture that spans an entire video. Alongwith an image-based rendering module, these trajectories can be used for anumber of downstream applications, such as turning still images into seamlesslylooping videos, or allowing users to realistically interact with objects inreal pictures by interpreting the spectral volumes as image-space modal bases,which approximate object dynamics.