Abstract
We present a method for estimating detailed scene illumination using humanfaces in a single image. In contrast to previous works that estimate lightingin terms of low-order basis functions or distant point lights, our techniqueestimates illumination at a higher precision in the form of a non-parametricenvironment map. Based on the observation that faces can exhibit stronghighlight reflections from a broad range of lighting directions, we propose adeep neural network for extracting highlights from faces, and then trace thesereflections back to the scene to acquire the environment map. Since realtraining data for highlight extraction is very limited, we introduce anunsupervised scheme for finetuning the network on real images, based on theconsistent diffuse chromaticity of a given face seen in multiple real images.In tracing the estimated highlights to the environment, we reduce the blurringeffect of skin reflectance on reflected light through a deconvolutiondetermined by prior knowledge on face material properties. Comparisons toprevious techniques for highlight extraction and illumination estimation showthe state-of-the-art performance of this approach on a variety of indoor andoutdoor scenes.