Barbershop: GAN-based Image Compositing using Segmentation Masks

  • 2021-06-02 23:20:43
  • Peihao Zhu, Rameen Abdal, John Femiani, Peter Wonka
  • 33

Abstract

Seamlessly blending features from multiple images is extremely challengingbecause of complex relationships in lighting, geometry, and partial occlusionwhich cause coupling between different parts of the image. Even though recentwork on GANs enables synthesis of realistic hair or faces, it remains difficultto combine them into a single, coherent, and plausible image rather than adisjointed set of image patches. We present a novel solution to image blending,particularly for the problem of hairstyle transfer, based on GAN-inversion. Wepropose a novel latent space for image blending which is better at preservingdetail and encoding spatial information, and propose a new GAN-embeddingalgorithm which is able to slightly modify images to conform to a commonsegmentation mask. Our novel representation enables the transfer of the visualproperties from multiple reference images including specific details such asmoles and wrinkles, and because we do image blending in a latent-space we areable to synthesize images that are coherent. Our approach avoids blendingartifacts present in other approaches and finds a globally consistent image.Our results demonstrate a significant improvement over the current state of theart in a user study, with users preferring our blending solution over 95percent of the time.

 

Quick Read (beta)

loading the full paper ...