ARF: Artistic Radiance Fields

  • 2022-06-13 18:55:31
  • Kai Zhang, Nick Kolkin, Sai Bi, Fujun Luan, Zexiang Xu, Eli Shechtman, Noah Snavely
  • 84

Abstract

We present a method for transferring the artistic features of an arbitrarystyle image to a 3D scene. Previous methods that perform 3D stylization onpoint clouds or meshes are sensitive to geometric reconstruction errors forcomplex real-world scenes. Instead, we propose to stylize the more robustradiance field representation. We find that the commonly used Gram matrix-basedloss tends to produce blurry results without faithful brushstrokes, andintroduce a nearest neighbor-based loss that is highly effective at capturingstyle details while maintaining multi-view consistency. We also propose a noveldeferred back-propagation method to enable optimization of memory-intensiveradiance fields using style losses defined on full-resolution rendered images.Our extensive evaluation demonstrates that our method outperforms baselines bygenerating artistic appearance that more closely resembles the style image.Please check our project page for video results and open-sourceimplementations: https://www.cs.cornell.edu/projects/arf/ .

 

Quick Read (beta)

loading the full paper ...