Language Modeling with Generative AdversarialNetworks

  • 2018-04-08 03:18:13
  • Mehrad Moradshahi, Utkarsh Contractor
  • 3

Abstract

Generative Adversarial Networks (GANs) have been promising in the field ofimage generation, however, they have been hard to train for languagegeneration. GANs were originally designed to output differentiable values, sodiscrete language generation is challenging for them which causes high levelsof instability in training GANs. Consequently, past work has resorted topre-training with maximum-likelihood or training GANs without pre-training witha WGAN objective with a gradient penalty. In this study, we present acomparison of those approaches. Furthermore, we present the results of someexperiments that indicate better training and convergence of Wasserstein GANs(WGANs) when a weaker regularization term is enforcing the Lipschitzconstraint.

 

Quick Read (beta)

loading the full paper ...