LaFIn: Generative Landmark Guided Face Inpainting

  • 2019-11-26 08:20:50
  • Yang Yang, Xiaojie Guo, Jiayi Ma, Lin Ma, Haibin Ling
  • 22

Abstract

It is challenging to inpaint face images in the wild, due to the largevariation of appearance, such as different poses, expressions and occlusions. Agood inpainting algorithm should guarantee the realism of output, including thetopological structure among eyes, nose and mouth, as well as the attributeconsistency on pose, gender, ethnicity, expression, etc. This paper studies aneffective deep learning based strategy to deal with these issues, whichcomprises of a facial landmark predicting subnet and an image inpaintingsubnet. Concretely, given partial observation, the landmark predictor aims toprovide the structural information (e.g. topological relationship andexpression) of incomplete faces, while the inpaintor is to generate plausibleappearance (e.g. gender and ethnicity) conditioned on the predicted landmarks.Experiments on the CelebA-HQ and CelebA datasets are conducted to reveal theefficacy of our design and, to demonstrate its superiority overstate-of-the-art alternatives both qualitatively and quantitatively. Inaddition, we assume that high-quality completed faces together with theirlandmarks can be utilized as augmented data to further improve the performanceof (any) landmark predictor, which is corroborated by experimental results onthe 300W and WFLW datasets.

 

Quick Read (beta)

loading the full paper ...