Abstract
We present a method for retiming people in an ordinary, naturalvideo---manipulating and editing the time in which different motions ofindividuals in the video occur. We can temporally align different motions,change the speed of certain actions (speeding up/slowing down, or entirely"freezing" people), or "erase" selected people from the video altogether. Weachieve these effects computationally via a dedicated learning-based layeredvideo representation, where each frame in the video is decomposed into separateRGBA layers, representing the appearance of different people in the video. Akey property of our model is that it not only disentangles the direct motionsof each person in the input video, but also correlates each personautomatically with the scene changes they generate---e.g., shadows,reflections, and motion of loose clothing. The layers can be individuallyretimed and recombined into a new video, allowing us to achieve realistic,high-quality renderings of retiming effects for real-world videos depictingcomplex actions and involving multiple individuals, including dancing,trampoline jumping, or group running.