Abstract
Large language models (LLMs) demonstrate promising translation performanceamong various natural languages. However, many LLMs especially the open-sourcedones, such as BLOOM and LLaMA, are English-dominant and support only dozens ofnatural languages, making the potential of LLMs on language translation lessexplored. In this work, we present BigTrans which adapts LLaMA that covers only20 languages and enhances it with multilingual translation capability on morethan 100 languages. BigTrans is built upon LLaMA-13B and it is optimized inthree steps. First, we continue training LLaMA with massive Chinese monolingualdata. Second, we continue training the model with a large-scale paralleldataset that covers 102 natural languages. Third, we instruct-tune thefoundation model with multilingual translation instructions, leading to ourBigTrans model. The preliminary experiments on multilingual translation showthat BigTrans performs comparably with ChatGPT and Google Translate in manylanguages and even outperforms ChatGPT in 8 language pairs. We release theBigTrans model and hope it can advance the research progress.