Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

  • 2021-06-03 12:57:54
  • Evgeniia Razumovskaia, Goran Glavaš, Olga Majewska, Edoardo M. Ponti, Anna Korhonen, Ivan Vulić
  • 0


In task-oriented dialogue (ToD), a user holds a conversation with anartificial agent to complete a concrete task. Although this technologyrepresents one of the central objectives of AI and has been the focus of evermore intense research and development efforts, it is currently limited to a fewnarrow domains (e.g., food ordering, ticket booking) and a handful of languages(e.g., English, Chinese). This work provides an extensive overview of existingmethods and resources in multilingual ToD as an entry point to this excitingand emerging field. We find that the most critical factor preventing thecreation of truly multilingual ToD systems is the lack of datasets in mostlanguages for both training and evaluation. In fact, acquiring annotations orhuman feedback for each component of modular systems or for data-hungryend-to-end systems is expensive and tedious. Hence, state-of-the-art approachesto multilingual ToD mostly rely on (zero- or few-shot) cross-lingual transferfrom resource-rich languages (almost exclusively English), either by means ofmachine translation or multilingual representations. These approaches arecurrently viable only for typologically similar languages and languages withparallel / monolingual corpora available. On the other hand, theireffectiveness beyond these boundaries is doubtful or hard to assess due to thelack of linguistically diverse benchmarks (especially for natural languagegeneration and end-to-end evaluation). To overcome this limitation, we drawparallels between components of the ToD pipeline and other NLP tasks, which caninspire solutions for learning in low-resource scenarios. Finally, we listadditional challenges that multilinguality poses for related areas (such asspeech and human-centred evaluation), and indicate future directions that holdpromise to further expand language coverage and dialogue capabilities ofcurrent ToD systems.


Quick Read (beta)

loading the full paper ...