Abstract
Automatically generating coherent and semantically meaningful text has manyapplications in machine translation, dialogue systems, image captioning, etc.Recently, by combining with policy gradient, Generative Adversarial Nets (GAN)that use a discriminative model to guide the training of the generative modelas a reinforcement learning policy has shown promising results in textgeneration. However, the scalar guiding signal is only available after theentire text has been generated and lacks intermediate information about textstructure during the generative process. As such, it limits its success whenthe length of the generated text samples is long (more than 20 words). In thispaper, we propose a new framework, called LeakGAN, to address the problem forlong text generation. We allow the discriminative net to leak its ownhigh-level extracted features to the generative net to further help theguidance. The generator incorporates such informative signals into allgeneration steps through an additional Manager module, which takes theextracted features of current generated words and outputs a latent vector toguide the Worker module for next-word generation. Our extensive experiments onsynthetic data and various real-world tasks with Turing test demonstrate thatLeakGAN is highly effective in long text generation and also improves theperformance in short text generation scenarios. More importantly, without anysupervision, LeakGAN would be able to implicitly learn sentence structures onlythrough the interaction between Manager and Worker.