The Unusual Effectiveness of Averaging in GAN Training

  • 2018-06-12 13:27:23
  • Yasin Yazıcı, Chuan-Sheng Foo, Stefan Winkler, Kim-Hui Yap, Georgios Piliouras, Vijay Chandrasekhar
  • 24

Abstract

We show empirically that the optimal strategy of parameter averaging in aminmax convex-concave game setting is also strikingly effective in the nonconvex-concave GAN setting, specifically alleviating the convergence issuesassociated with cycling behavior observed in GANs. We show that averaging overgenerator parameters outside of the trainig loop consistently improvesinception and FID scores on different architectures and for different GANobjectives. We provide comprehensive experimental results across a range ofdatasets, bilinear games, mixture of Gaussians, CIFAR-10, STL-10, CelebA andImageNet, to demonstrate its effectiveness. We achieve state-of-the-art resultson CIFAR-10 and produce clean CelebA face images, demonstrating that averagingis one of the most effective techniques for training highly performant GANs.

 

Quick Read (beta)

loading the full paper ...