Rewriting Geometric Rules of a GAN

  • 2022-07-28 18:59:36
  • Sheng-Yu Wang, David Bau, Jun-Yan Zhu
  • 81

Abstract

Deep generative models make visual content creation more accessible to noviceusers by automating the synthesis of diverse, realistic content based on acollected dataset. However, the current machine learning approaches miss a keyelement of the creative process -- the ability to synthesize things that go farbeyond the data distribution and everyday experience. To begin to address thisissue, we enable a user to "warp" a given model by editing just a handful oforiginal model outputs with desired geometric changes. Our method applies alow-rank update to a single model layer to reconstruct edited examples.Furthermore, to combat overfitting, we propose a latent space augmentationmethod based on style-mixing. Our method allows a user to create a model thatsynthesizes endless objects with defined geometric changes, enabling thecreation of a new generative model without the burden of curating a large-scaledataset. We also demonstrate that edited models can be composed to achieveaggregated effects, and we present an interactive interface to enable users tocreate new models through composition. Empirical measurements on multiple testcases suggest the advantage of our method against recent GAN fine-tuningmethods. Finally, we showcase several applications using the edited models,including latent space interpolation and image editing.

 

Quick Read (beta)

loading the full paper ...