Zero-Shot Cross-Lingual Transfer with Meta Learning

  • 2020-04-02 14:40:52
  • Farhad Nooralahzadeh, Giannis Bekoulis, Johannes Bjerva, Isabelle Augenstein
  • 0

Abstract

Learning what to share between tasks has been a topic of great importancerecently, as strategic sharing of knowledge has been shown to improve theperformance of downstream tasks. In multilingual applications, sharing ofknowledge between languages is important when considering the fact that mostlanguages in the world suffer from being under-resourced. In this paper, weconsider the setting of training models on multiple different languages at thesame time, when English training data, but little or no in-language data isavailable. We show that this challenging setup can be approached usingmeta-learning, where, in addition to training a source language model, anothermodel learns to select which training instances are the most beneficial. Weexperiment using standard supervised, zero-shot cross-lingual, as well asfew-shot cross-lingual settings for different natural language understandingtasks (natural language inference, question answering). Our extensiveexperimental setup demonstrates the consistent effectiveness of meta-learningin a total of 16 languages. We improve upon the state-of-the-art for zero-shotand few-shot NLI and QA tasks on two NLI datasets (i.e., MultiNLI and XNLI),and on the X-WikiRE dataset, respectively. We further conduct a comprehensiveanalysis, which indicates that the correlation of typological features betweenlanguages can further explain when parameter sharing learned via meta-learningis beneficial.

 

Quick Read (beta)

loading the full paper ...