Abstract
We conduct an empirical study of neural machine translation (NMT) for trulylow-resource languages, and propose a training curriculum fit for cases whenboth parallel training data and compute resource are lacking, reflecting thereality of most of the world's languages and the researchers working on theselanguages. Previously, unsupervised NMT, which employs back-translation (BT)and auto-encoding (AE) tasks has been shown barren for low-resource languages.We demonstrate that leveraging comparable data and code-switching as weaksupervision, combined with BT and AE objectives, result in remarkableimprovements for low-resource languages even when using only modest computeresources. The training curriculum proposed in this work achieves BLEU scoresthat improve over supervised NMT trained on the same backbone architecture by+12.2 BLEU for English to Gujarati and +3.7 BLEU for English to Kazakh,showcasing the potential of weakly-supervised NMT for the low-resourcelanguages. When trained on supervised data, our training curriculum achieves anew state-of-the-art result on the Somali dataset (BLEU of 29.3 for Somali toEnglish). We also observe that adding more time and GPUs to training canfurther improve performance, which underscores the importance of reportingcompute resource usage in MT research.