Abstract
Multilingual neural machine translation (NMT) enables training a single modelthat supports translation from multiple source languages into multiple targetlanguages. In this paper, we push the limits of multilingual NMT in terms ofnumber of languages being used. We perform extensive experiments in trainingmassively multilingual NMT models, translating up to 102 languages to and fromEnglish within a single model. We explore different setups for training suchmodels and analyze the trade-offs between translation quality and variousmodeling decisions. We report results on the publicly available TED talksmultilingual corpus where we show that massively multilingual many-to-manymodels are effective in low resource settings, outperforming the previousstate-of-the-art while supporting up to 59 languages. Our experiments on alarge-scale dataset with 102 languages to and from English and up to onemillion examples per direction also show promising results, surpassing strongbilingual baselines and encouraging future work on massively multilingual NMT.