Choosing Transfer Languages for Cross-Lingual Learning

  • 2019-05-29 19:19:47
  • Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, Graham Neubig
  • 7


Cross-lingual transfer, where a high-resource transfer language is used toimprove the accuracy of a low-resource task language, is now an invaluable toolfor improving performance of natural language processing (NLP) on low-resourcelanguages. However, given a particular task language, it is not clear whichlanguage to transfer from, and the standard strategy is to select languagesbased on ad hoc criteria, usually the intuition of the experimenter. Since alarge number of features contribute to the success of cross-lingual transfer(including phylogenetic similarity, typological properties, lexical overlap, orsize of available data), even the most enlightened experimenter rarelyconsiders all these factors for the particular task at hand. In this paper, weconsider this task of automatically selecting optimal transfer languages as aranking problem, and build models that consider the aforementioned features toperform this prediction. In experiments on representative NLP tasks, wedemonstrate that our model predicts good transfer languages much better than adhoc baselines considering single features in isolation, and glean insights onwhat features are most informative for each different NLP tasks, which mayinform future ad hoc selection even without use of our method. Code, data, andpre-trained models are available at


Quick Read (beta)

loading the full paper ...