StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery

  • 2021-03-31 17:51:25
  • Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, Dani Lischinski
  • 266

Abstract

Inspired by the ability of StyleGAN to generate highly realistic images in avariety of domains, much recent work has focused on understanding how to usethe latent spaces of StyleGAN to manipulate generated and real images. However,discovering semantically meaningful latent manipulations typically involvespainstaking human examination of the many degrees of freedom, or an annotatedcollection of images for each desired manipulation. In this work, we exploreleveraging the power of recently introduced Contrastive Language-ImagePre-training (CLIP) models in order to develop a text-based interface forStyleGAN image manipulation that does not require such manual effort. We firstintroduce an optimization scheme that utilizes a CLIP-based loss to modify aninput latent vector in response to a user-provided text prompt. Next, wedescribe a latent mapper that infers a text-guided latent manipulation step fora given input image, allowing faster and more stable text-based manipulation.Finally, we present a method for mapping a text prompts to input-agnosticdirections in StyleGAN's style space, enabling interactive text-driven imagemanipulation. Extensive results and comparisons demonstrate the effectivenessof our approaches.

 

Quick Read (beta)

loading the full paper ...