Zero-Resource Multilingual Model Transfer: Learning What to Share

  • 2018-10-08 16:11:01
  • Xilun Chen, Ahmed Hassan Awadallah, Hany Hassan, Wei Wang, Claire Cardie
  • 7

Abstract

Modern natural language processing and understanding applications haveenjoyed a great boost utilizing neural networks models. However, this is notthe case for most languages especially low-resource ones with insufficientannotated training data. Cross-lingual transfer learning methods improve theperformance on a low-resource target language by leveraging labeled data fromother (source) languages, typically with the help of cross-lingual resourcessuch as parallel corpora. In this work, we propose the first zero-resourcemultilingual transfer learning model that can utilize training data in multiplesource languages, while not requiring target language training data norcross-lingual supervision. Unlike existing methods that only rely onlanguage-invariant features for cross-lingual transfer, our approach utilizesboth language-invariant and language-specific features in a coherent way. Ourmodel leverages adversarial networks to learn language-invariant features andmixture-of-experts models to dynamically exploit the relation between thetarget language and each individual source language. This enables our model tolearn effectively what to share between various languages in the multilingualsetup. It results in significant performance gains over prior art, as shown inan extensive set of experiments over multiple text classification and sequencetagging tasks including a large-scale real-world industry dataset.

 

Quick Read (beta)

loading the full paper ...