Abstract
We introduce our efforts towards building a universal neural machinetranslation (NMT) system capable of translating between any language pair. Weset a milestone towards this goal by building a single massively multilingualNMT model handling 103 languages trained on over 25 billion examples. Oursystem demonstrates effective transfer learning ability, significantlyimproving translation quality of low-resource languages, while keepinghigh-resource language translation quality on-par with competitive bilingualbaselines. We provide in-depth analysis of various aspects of model buildingthat are crucial to achieving quality and practicality in universal NMT. Whilewe prototype a high-quality universal translation system, our extensiveempirical analysis exposes issues that need to be further addressed, and wesuggest directions for future research.