MixSyn: Learning Composition and Style for Multi-Source Image Synthesis

  • 2021-11-24 18:58:34
  • Ilke Demir, Umur A. Ciftci
  • 1

Abstract

Synthetic images created by generative models increase in quality andexpressiveness as newer models utilize larger datasets and novel architectures.Although this photorealism is a positive side-effect from a creativestandpoint, it becomes problematic when such generative models are used forimpersonation without consent. Most of these approaches are built on thepartial transfer between source and target pairs, or they generate completelynew samples based on an ideal distribution, still resembling the closest realsample in the dataset. We propose MixSyn (read as " mixin' ") for learningnovel fuzzy compositions from multiple sources and creating novel images as amix of image regions corresponding to the compositions. MixSyn not onlycombines uncorrelated regions from multiple source masks into a coherentsemantic composition, but also generates mask-aware high qualityreconstructions of non-existing images. We compare MixSyn to state-of-the-artsingle-source sequential generation and collage generation approaches in termsof quality, diversity, realism, and expressive power; while also showcasinginteractive synthesis, mix & match, and edit propagation tasks, with no maskdependency.

 

Quick Read (beta)

loading the full paper ...