Abstract
Numerous recent work on unsupervised machine translation (UMT) implies thatcompetent unsupervised translations of low-resource and unrelated languages,such as Nepali or Sinhala, are only possible if the model is trained in amassive multilingual environment, where theses low-resource languages are mixedwith high-resource counterparts. Nonetheless, while the high-resource languagesgreatly help kick-start the target low-resource translation tasks, the languagediscrepancy between them may hinder their further improvement. In this work, wepropose a simple refinement procedure to disentangle languages from apre-trained multilingual UMT model for it to focus on only the targetlow-resource task. Our method achieves the state of the art in the fullyunsupervised translation tasks of English to Nepali, Sinhala, Gujarati,Latvian, Estonian and Kazakh, with BLEU score gains of 3.5, 3.5, 3.3, 4.1, 4.2,and 3.3, respectively. Our codebase is available athttps://github.com/nxphi47/refine_unsup_multilingual_mt