Abstract
We translate a closed text that is known in advance into a severely lowresource language by leveraging massive source parallelism. In other words,given a text in 124 source languages, we translate it into a severely lowresource language using only ~1,000 lines of low resource data without anyexternal help. Firstly, we propose a systematic method to rank and choosesource languages that are close to the low resource language. We call thelinguistic definition of language family Family of Origin (FAMO), and we callthe empirical definition of higher-ranked languages using our metrics Family ofChoice (FAMC). Secondly, we build an Iteratively Pretrained MultilingualOrder-preserving Lexiconized Transformer (IPML) to train on ~1,000 lines(~3.5%) of low resource data. To translate named entities correctly, we build amassive lexicon table for 2,939 Bible named entities in 124 source languages,and include many that occur once and covers more than 66 severely low resourcelanguages. Moreover, we also build a novel method of combining translationsfrom different source languages into one. Using English as a hypothetical lowresource language, we get a +23.9 BLEU increase over a multilingual baseline,and a +10.3 BLEU increase over our asymmetric baseline in the Bible dataset. Weget a 42.8 BLEU score for Portuguese-English translation on the medical EMEAdataset. We also have good results for a real severely low resource Mayanlanguage, Eastern Pokomchi.