StyleGAN knows Normal, Depth, Albedo, and More

  • 2023-06-01 18:59:57
  • Anand Bhattad, Daniel McKee, Derek Hoiem, D. A. Forsyth
  • 0


Intrinsic images, in the original sense, are image-like maps of sceneproperties like depth, normal, albedo or shading. This paper demonstrates thatStyleGAN can easily be induced to produce intrinsic images. The procedure isstraightforward. We show that, if StyleGAN produces $G({w})$ from latents${w}$, then for each type of intrinsic image, there is a fixed offset ${d}_c$so that $G({w}+{d}_c)$ is that type of intrinsic image for $G({w})$. Here${d}_c$ is {\em independent of ${w}$}. The StyleGAN we used was pretrained byothers, so this property is not some accident of our training regime. We showthat there are image transformations StyleGAN will {\em not} produce in thisfashion, so StyleGAN is not a generic image regression engine. It is conceptually exciting that an image generator should ``know'' andrepresent intrinsic images. There may also be practical advantages to using agenerative model to produce intrinsic images. The intrinsic images obtainedfrom StyleGAN compare well both qualitatively and quantitatively with thoseobtained by using SOTA image regression techniques; but StyleGAN's intrinsicimages are robust to relighting effects, unlike SOTA methods.


Quick Read (beta)

loading the full paper ...