DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

  • 2019-11-01 18:16:54
  • Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan
  • 19


We present a large, tunable neural conversational response generation model,DialoGPT (dialogue generative pre-trained transformer). Trained on 147Mconversation-like exchanges extracted from Reddit comment chains over a periodspanning from 2005 through 2017, DialoGPT extends the Hugging Face PyTorchtransformer to attain a performance close to human both in terms of automaticand human evaluation in single-turn dialogue settings. We show thatconversational systems that leverage DialoGPT generate more relevant,contentful and context-consistent responses than strong baseline systems. Thepre-trained model and training pipeline are publicly released to facilitateresearch into neural response generation and the development of moreintelligent open-domain dialogue systems.


Quick Read (beta)

loading the full paper ...