StyleLight: HDR Panorama Generation for Lighting Estimation and Editing

  • 2022-07-29 18:58:58
  • Guangcong Wang, Yinuo Yang, Chen Change Loy, Ziwei Liu
  • 25

Abstract

We present a new lighting estimation and editing framework to generatehigh-dynamic-range (HDR) indoor panorama lighting from a single limitedfield-of-view (LFOV) image captured by low-dynamic-range (LDR) cameras.Existing lighting estimation methods either directly regress lightingrepresentation parameters or decompose this problem into LFOV-to-panorama andLDR-to-HDR lighting generation sub-tasks. However, due to the partialobservation, the high-dynamic-range lighting, and the intrinsic ambiguity of ascene, lighting estimation remains a challenging task. To tackle this problem,we propose a coupled dual-StyleGAN panorama synthesis network (StyleLight) thatintegrates LDR and HDR panorama synthesis into a unified framework. The LDR andHDR panorama synthesis share a similar generator but have separatediscriminators. During inference, given an LDR LFOV image, we propose afocal-masked GAN inversion method to find its latent code by the LDR panoramasynthesis branch and then synthesize the HDR panorama by the HDR panoramasynthesis branch. StyleLight takes LFOV-to-panorama and LDR-to-HDR lightinggeneration into a unified framework and thus greatly improves lightingestimation. Extensive experiments demonstrate that our framework achievessuperior performance over state-of-the-art methods on indoor lightingestimation. Notably, StyleLight also enables intuitive lighting editing onindoor HDR panoramas, which is suitable for real-world applications. Code isavailable at https://style-light.github.io.

 

Quick Read (beta)

loading the full paper ...