Abstract
Face Restoration (FR) is a crucial area within image and video processing,focusing on reconstructing high-quality portraits from degraded inputs. Despiteadvancements in image FR, video FR remains relatively under-explored, primarilydue to challenges related to temporal consistency, motion artifacts, and thelimited availability of high-quality video data. Moreover, traditional facerestoration typically prioritizes enhancing resolution and may not give as muchconsideration to related tasks such as facial colorization and inpainting. Inthis paper, we propose a novel approach for the Generalized Video FaceRestoration (GVFR) task, which integrates video BFR, inpainting, andcolorization tasks that we empirically show to benefit each other. We present aunified framework, termed as stable video face restoration (SVFR), whichleverages the generative and motion priors of Stable Video Diffusion (SVD) andincorporates task-specific information through a unified face restorationframework. A learnable task embedding is introduced to enhance taskidentification. Meanwhile, a novel Unified Latent Regularization (ULR) isemployed to encourage the shared feature representation learning amongdifferent subtasks. To further enhance the restoration quality and temporalstability, we introduce the facial prior learning and the self-referredrefinement as auxiliary strategies used for both training and inference. Theproposed framework effectively combines the complementary strengths of thesetasks, enhancing temporal coherence and achieving superior restoration quality.This work advances the state-of-the-art in video FR and establishes a newparadigm for generalized video face restoration. Code and video demo areavailable at https://github.com/wangzhiyaoo/SVFR.git.