Competence-based Curriculum Learning for Multilingual Machine Translation

  • 2021-09-09 02:52:34
  • Mingliang Zhang, Fandong Meng, Yunhai Tong, Jie Zhou
  • 3


Currently, multilingual machine translation is receiving more and moreattention since it brings better performance for low resource languages (LRLs)and saves more space. However, existing multilingual machine translation modelsface a severe challenge: imbalance. As a result, the translation performance ofdifferent languages in multilingual translation models are quite different. Weargue that this imbalance problem stems from the different learningcompetencies of different languages. Therefore, we focus on balancing thelearning competencies of different languages and propose Competence-basedCurriculum Learning for Multilingual Machine Translation, named CCL-M.Specifically, we firstly define two competencies to help schedule the highresource languages (HRLs) and the low resource languages: 1) Self-evaluatedCompetence, evaluating how well the language itself has been learned; and 2)HRLs-evaluated Competence, evaluating whether an LRL is ready to be learnedaccording to HRLs' Self-evaluated Competence. Based on the above competencies,we utilize the proposed CCL-M algorithm to gradually add new languages into thetraining set in a curriculum learning manner. Furthermore, we propose a novelcompetenceaware dynamic balancing sampling strategy for better selectingtraining samples in multilingual training. Experimental results show that ourapproach has achieved a steady and significant performance gain compared to theprevious state-of-the-art approach on the TED talks dataset.


Quick Read (beta)

loading the full paper ...