Attention-Informed Mixed-Language Training for Zero-shot Cross-lingual Task-oriented Dialogue Systems

  • 2019-11-21 03:52:50
  • Zihan Liu, Genta Indra Winata, Zhaojiang Lin, Peng Xu, Pascale Fung
  • 4

Abstract

Recently, data-driven task-oriented dialogue systems have achieved promisingperformance in English. However, developing dialogue systems that supportlow-resource languages remains a long-standing challenge due to the absence ofhigh-quality data. In order to circumvent the expensive and time-consuming datacollection, we introduce Attention-Informed Mixed-Language Training (MLT), anovel zero-shot adaptation method for cross-lingual task-oriented dialoguesystems. It leverages very few task-related parallel word pairs to generatecode-switching sentences for learning the inter-lingual semantics acrosslanguages. Instead of manually selecting the word pairs, we propose to extractsource words based on the scores computed by the attention layer of a trainedEnglish task-related model and then generate word pairs using existingbilingual dictionaries. Furthermore, intensive experiments with differentcross-lingual embeddings demonstrate the effectiveness of our approach.Finally, with very few word pairs, our model achieves significant zero-shotadaptation performance improvements in both cross-lingual dialogue statetracking and natural language understanding (i.e., intent detection and slotfilling) tasks compared to the current state-of-the-art approaches, whichutilize a much larger amount of bilingual data.

 

Quick Read (beta)

loading the full paper ...