Abstract
We introduce SynthLight, a diffusion model for portrait relighting. Ourapproach frames image relighting as a re-rendering problem, where pixels aretransformed in response to changes in environmental lighting conditions. Usinga physically-based rendering engine, we synthesize a dataset to simulate thislighting-conditioned transformation with 3D head assets under varying lighting.We propose two training and inference strategies to bridge the gap between thesynthetic and real image domains: (1) multi-task training that takes advantageof real human portraits without lighting labels; (2) an inference timediffusion sampling procedure based on classifier-free guidance that leveragesthe input portrait to better preserve details. Our method generalizes todiverse real photographs and produces realistic illumination effects, includingspecular highlights and cast shadows, while preserving the subject's identity.Our quantitative experiments on Light Stage data demonstrate results comparableto state-of-the-art relighting methods. Our qualitative results on in-the-wildimages showcase rich and unprecedented illumination effects. Project Page:\url{https://vrroom.github.io/synthlight/}