3D Cinemagraphy from a Single Image

  • 2023-03-10 06:08:23
  • Xingyi Li, Zhiguo Cao, Huiqiang Sun, Jianming Zhang, Ke Xian, Guosheng Lin
  • 72


We present 3D Cinemagraphy, a new technique that marries 2D image animationwith 3D photography. Given a single still image as input, our goal is togenerate a video that contains both visual content animation and camera motion.We empirically find that naively combining existing 2D image animation and 3Dphotography methods leads to obvious artifacts or inconsistent animation. Ourkey insight is that representing and animating the scene in 3D space offers anatural solution to this task. To this end, we first convert the input imageinto feature-based layered depth images using predicted depth values, followedby unprojecting them to a feature point cloud. To animate the scene, we performmotion estimation and lift the 2D motion into the 3D scene flow. Finally, toresolve the problem of hole emergence as points move forward, we propose tobidirectionally displace the point cloud as per the scene flow and synthesizenovel views by separately projecting them into target image planes and blendingthe results. Extensive experiments demonstrate the effectiveness of our method.A user study is also conducted to validate the compelling rendering results ofour method.


Quick Read (beta)

loading the full paper ...