Semantic Pyramid for Image Generation

  • 2020-03-13 12:23:37
  • Assaf Shocher, Yossi Gandelsmam, Inbar Mosseri, Michal Yarom, Michal Irani, William T. Freeman, Tali Dekel
  • 75

Abstract

We present a novel GAN-based model that utilizes the space of deep featureslearned by a pre-trained classification model. Inspired by classical imagepyramid representations, we construct our model as a Semantic GenerationPyramid -- a hierarchical framework which leverages the continuum of semanticinformation encapsulated in such deep features; this ranges from low levelinformation contained in fine features to high level, semantic informationcontained in deeper features. More specifically, given a set of featuresextracted from a reference image, our model generates diverse image samples,each with matching features at each semantic level of the classification model.We demonstrate that our model results in a versatile and flexible frameworkthat can be used in various classic and novel image generation tasks. Theseinclude: generating images with a controllable extent of semantic similarity toa reference image, and different manipulation tasks such assemantically-controlled inpainting and compositing; all achieved with the samemodel, with no further training.

 

Quick Read (beta)

loading the full paper ...