Abstract
Data scarcity is a long-standing and crucial challenge that hinders quickdevelopment of task-oriented dialogue systems across multiple domains:task-oriented dialogue models are expected to learn grammar, syntax, dialoguereasoning, decision making, and language generation from absurdly small amountsof task-specific data. In this paper, we demonstrate that recent progress inlanguage modeling pre-training and transfer learning shows promise to overcomethis problem. We propose a task-oriented dialogue model that operates solely ontext input: it effectively bypasses explicit policy and language generationmodules. Building on top of the TransferTransfo framework (Wolf et al., 2019)and generative model pre-training (Radford et al., 2019), we validate theapproach on complex multi-domain task-oriented dialogues from the MultiWOZdataset. Our automatic and human evaluations show that the proposed model is onpar with a strong task-specific neural baseline. In the long run, our approachholds promise to mitigate the data scarcity problem, and to support theconstruction of more engaging and more eloquent task-oriented conversationalagents.