Self-Attention Generative Adversarial Networks

  • 2018-05-21 23:10:35
  • Han Zhang, Ian Goodfellow, Dimitris Metaxas, Augustus Odena
  • 184

Abstract

In this paper, we propose the Self-Attention Generative Adversarial Network(SAGAN) which allows attention-driven, long-range dependency modeling for imagegeneration tasks. Traditional convolutional GANs generate high-resolutiondetails as a function of only spatially local points in lower-resolutionfeature maps. In SAGAN, details can be generated using cues from all featurelocations. Moreover, the discriminator can check that highly detailed featuresin distant portions of the image are consistent with each other. Furthermore,recent work has shown that generator conditioning affects GAN performance.Leveraging this insight, we apply spectral normalization to the GAN generatorand find that this improves training dynamics. The proposed SAGAN achieves thestate-of-the-art results, boosting the best published Inception score from 36.8to 52.52 and reducing Frechet Inception distance from 27.62 to 18.65 on thechallenging ImageNet dataset. Visualization of the attention layers shows thatthe generator leverages neighborhoods that correspond to object shapes ratherthan local regions of fixed shape.

 

Quick Read (beta)

loading the full paper ...