Abstract
Unsupervised translation has reached impressive performance on resource-richlanguage pairs such as English-French and English-German. However, earlystudies have shown that in more realistic settings involving low-resource, rarelanguages, unsupervised translation performs poorly, achieving less than 3.0BLEU. In this work, we show that multilinguality is critical to makingunsupervised systems practical for low-resource settings. In particular, wepresent a single model for 5 low-resource languages (Gujarati, Kazakh, Nepali,Sinhala, and Turkish) to and from English directions, which leveragesmonolingual and auxiliary parallel data from other high-resource language pairsvia a three-stage training scheme. We outperform all current state-of-the-artunsupervised baselines for these languages, achieving gains of up to 14.4 BLEU.Additionally, we outperform a large collection of supervised WMT submissionsfor various language pairs as well as match the performance of the currentstate-of-the-art supervised model for Nepali-English. We conduct a series ofablation studies to establish the robustness of our model under differentdegrees of data quality, as well as to analyze the factors which led to thesuperior performance of the proposed approach over traditional unsupervisedmodels.