Abstract
This paper examines the problem of adapting neural machine translationsystems to new, low-resourced languages (LRLs) as effectively and rapidly aspossible. We propose methods based on starting with massively multilingual"seed models", which can be trained ahead-of-time, and then continuing trainingon data related to the LRL. We contrast a number of strategies, leading to anovel, simple, yet effective method of "similar-language regularization", wherewe jointly train on both a LRL of interest and a similar high-resourcedlanguage to prevent over-fitting to small LRL data. Experiments demonstratethat massively multilingual models, even without any explicit adaptation, aresurprisingly effective, achieving BLEU scores of up to 15.5 with no data fromthe LRL, and that the proposed similar-language regularization method improvesover other adaptation methods by 1.7 BLEU points average over 4 LRL settings.Code to reproduce experiments at https://github.com/neubig/rapid-adaptation