Small-GAN: Speeding Up GAN Training Using Core-sets

  • 2019-10-29 21:26:05
  • Samarth Sinha, Han Zhang, Anirudh Goyal, Yoshua Bengio, Hugo Larochelle, Augustus Odena
  • 60

Abstract

Recent work by Brock et al. (2018) suggests that Generative AdversarialNetworks (GANs) benefit disproportionately from large mini-batch sizes.Unfortunately, using large batches is slow and expensive on conventionalhardware. Thus, it would be nice if we could generate batches that wereeffectively large though actually small. In this work, we propose a method todo this, inspired by the use of Coreset-selection in active learning. Whentraining a GAN, we draw a large batch of samples from the prior and thencompress that batch using Coreset-selection. To create effectively largebatches of 'real' images, we create a cached dataset of Inception activationsof each training image, randomly project them down to a smaller dimension, andthen use Coreset-selection on those projected activations at training time. Weconduct experiments showing that this technique substantially reduces trainingtime and memory usage for modern GAN variants, that it reduces the fraction ofdropped modes in a synthetic dataset, and that it allows GANs to reach a newstate of the art in anomaly detection.

 

Quick Read (beta)

This feature is not avaialbe for this paper.