Abstract
We apply style transfer on mesh reconstructions of indoor scenes. Thisenables VR applications like experiencing 3D environments painted in the styleof a favorite artist. Style transfer typically operates on 2D images, makingstylization of a mesh challenging. When optimized over a variety of poses,stylization patterns become stretched out and inconsistent in size. On theother hand, model-based 3D style transfer methods exist that allow stylizationfrom a sparse set of images, but they require a network at inference time. Tothis end, we optimize an explicit texture for the reconstructed mesh of a sceneand stylize it jointly from all available input images. Our depth- andangle-aware optimization leverages surface normal and depth data of theunderlying mesh to create a uniform and consistent stylization for the wholescene. Our experiments show that our method creates sharp and detailed resultsfor the complete scene without view-dependent artifacts. Through extensiveablation studies, we show that the proposed 3D awareness enables style transferto be applied to the 3D domain of a mesh. Our method can be used to render astylized mesh in real-time with traditional rendering pipelines.