Improving Zero-shot Multilingual Neural Machine Translation for Low-Resource Languages

  • 2021-10-02 02:50:53
  • Chenyang Li, Gongxu Luo
  • 0


Although the multilingual Neural Machine Translation(NMT), which extendsGoogle's multilingual NMT, has ability to perform zero-shot translation and theiterative self-learning algorithm can improve the quality of zero-shottranslation, it confronts with two problems: the multilingual NMT model isprone to generate wrong target language when implementing zero-shottranslation; the self-learning algorithm, which uses beam search to generatesynthetic parallel data, demolishes the diversity of the generated sourcelanguage and amplifies the impact of the same noise during the iterativelearning process. In this paper, we propose the tagged-multilingual NMT modeland improve the self-learning algorithm to handle these two problems. Firstly,we extend the Google's multilingual NMT model and add target tokens to thetarget languages, which associates the start tag with the target language toensure that the source language can be translated to the required targetlanguage. Secondly, we improve the self-learning algorithm by replacing beamsearch with random sample to increases the diversity of the generated data andmakes it properly cover the true data distribution. Experimental results onIWSLT show that the adjusted tagged-multilingual NMT separately obtains 9.41and 7.85 BLEU scores over the multilingual NMT on 2010 and 2017Romanian-Italian test sets. Similarly, it obtains 9.08 and 7.99 BLEU scores onItalian-Romanian zero-shot translation. Furthermore, the improved self-learningalgorithm shows its superiorities over the conventional self-learning algorithmon zero-shot translations.


Quick Read (beta)

loading the full paper ...