MixNMatch: Multifactor Disentanglement and Encodingfor Conditional Image Generation

  • 2019-11-26 18:49:39
  • Yuheng Li, Krishna Kumar Singh, Utkarsh Ojha, Yong Jae Lee
  • 68

Abstract

We present MixNMatch, a conditional generative model that learns todisentangle and encode background, object pose, shape, and texture from realimages with minimal supervision, for mix-and-match image generation. We buildupon FineGAN, an unconditional generative model, to learn the desireddisentanglement and image generator, and leverage adversarial joint image-codedistribution matching to learn the latent factor encoders. MixNMatch requiresbounding boxes during training to model background, but requires no othersupervision. Through extensive experiments, we demonstrate MixNMatch's abilityto accurately disentangle, encode, and combine multiple factors formix-and-match image generation, including sketch2color, cartoon2img, andimg2gif applications. Our code/models/demo can be found athttps://github.com/Yuheng-Li/MixNMatch

 

Quick Read (beta)

loading the full paper ...