3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image

  • 2021-11-30 23:27:10
  • Fangzhou Mu, Jian Wang, Yicheng Wu, Yin Li
  • 26

Abstract

Visual content creation has spurred a soaring interest given its applicationsin mobile photography and AR / VR. Style transfer and single-image 3Dphotography as two representative tasks have so far evolved independently. Inthis paper, we make a connection between the two, and address the challengingtask of 3D photo stylization - generating stylized novel views from a singleimage given an arbitrary style. Our key intuition is that style transfer andview synthesis have to be jointly modeled for this task. To this end, wepropose a deep model that learns geometry-aware content features forstylization from a point cloud representation of the scene, resulting inhigh-quality stylized images that are consistent across views. Further, weintroduce a novel training protocol to enable the learning using only 2Dimages. We demonstrate the superiority of our method via extensive qualitativeand quantitative studies, and showcase key applications of our method in lightof the growing demand for 3D content creation from 2D image assets.

 

Quick Read (beta)

loading the full paper ...