Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study

  • 2025-02-05 07:48:33
  • Menglong Cui, Pengzhi Gao, Wei Liu, Jian Luan, Bin Wang
  • 0

Abstract

Large language models (LLMs) have shown continuously improving multilingualcapabilities, and even small-scale open-source models have demonstrated rapidperformance enhancement. In this paper, we systematically explore the abilitiesof open LLMs with less than ten billion parameters to handle multilingualmachine translation (MT) tasks. We conduct comprehensive evaluations on sixpopular LLMs and find that models like Gemma2-9B exhibit impressivemultilingual translation capabilities. We then introduce the Parallel-FirstMonolingual-Second (PFMS) data mixing strategy in the continual pretrainingstage to further enhance the MT performance and present GemmaX2-28, a 9B modelachieving top-tier multilingual translation performance across 28 languages.Specifically, GemmaX2-28 consistently outperforms the state-of-the-art (SOTA)models such as TowerInstruct and XALMA and achieves competitive performancewith Google Translate and GPT-4-turbo.

 

Quick Read (beta)

loading the full paper ...