Towards an Understanding of Our World by GANing Videos in the Wild

  • 2017-11-30 14:55:16
  • Bernhard Kratzwald, Zhiwu Huang, Danda Pani Paudel, Luc Van Gool
  • 73

Abstract

Existing generative video models work well only for videos with a staticbackground. For dynamic scenes, applications of these models demand an extrapre-processing step of background stabilization. In fact, the task ofbackground stabilization may very often prove impossible for videos in thewild. To the best of our knowledge, we present the first video generationframework that works in the wild, without making any assumption on the videos'content. This allows us to avoid the background stabilization step, completely.The proposed method also outperforms the state-of-the-art methods even when thestatic background assumption is valid. This is achieved by designing a robustone-stream video generation architecture by exploiting Wasserstein GANframeworks for better convergence. Since the proposed architecture isone-stream, which does not formally distinguish between fore- and background,it can generate - and learn from - videos with dynamic backgrounds. Thesuperiority of our model is demonstrated by successfully applying it to threechallenging problems: video colorization, video inpainting, and futureprediction.

 

Introduction (beta)

None

 

Conclusion (beta)

None