A Universal Parent Model for Low-Resource Neural Machine Translation Transfer

  • 2019-09-14 03:11:52
  • Mozhdeh Gheini, Jonathan May
  • 1

Abstract

Transfer learning from a high-resource language pair `parent' has been provento be an effective way to improve neural machine translation quality forlow-resource language pairs `children.' However, previous approaches build acustom parent model or at least update an existing parent model's vocabularyfor each child language pair they wish to train, in an effort to align parentand child vocabularies. This is not a practical solution. It is wasteful todevote the majority of training time for new language pairs to optimizingparameters on an unrelated data set. Further, this overhead reduces the utilityof neural machine translation for deployment in humanitarian assistancescenarios, where extra time to deploy a new language pair can mean thedifference between life and death. In this work, we present a `universal'pre-trained neural parent model with constant vocabulary that can be used as astarting point for training practically any new low-resource language to afixed target language. We demonstrate that our approach, which leveragesorthography unification and a broad-coverage approach to subwordidentification, generalizes well to several languages from a variety offamilies, and that translation systems built with our approach can be builtmore quickly than competing methods and with better quality as well.

 

Quick Read (beta)

loading the full paper ...