Adversarial Text Generation via Feature-Mover's Distance

  • 2018-09-17 16:03:13
  • Liqun Chen, Shuyang Dai, Chenyang Tao, Dinghan Shen, Zhe Gan, Haichao Zhang, Yizhe Zhang, Lawrence Carin
  • 5

Abstract

Generative adversarial networks (GANs) have achieved significant success ingenerating real-valued data. However, the discrete nature of text hinders theapplication of GAN to text-generation tasks. Instead of using the standard GANobjective, we propose to improve text-generation GAN via a novel approachinspired by optimal transport. Specifically, we consider matching the latentfeature distributions of real and synthetic sentences using a novel metric,termed the feature-mover's distance (FMD). This formulation leads to a highlydiscriminative critic and easy-to-optimize objective, overcoming themode-collapsing and brittle-training problems in existing methods. Extensiveexperiments are conducted on a variety of tasks to evaluate the proposed modelempirically, including unconditional text generation, style transfer fromnon-parallel text, and unsupervised cipher cracking. The proposed model yieldssuperior performance, demonstrating wide applicability and effectiveness.

 

Quick Read (beta)

loading the full paper ...