Abstract
Recent advancement in code understanding and generation demonstrates thatcode LLMs fine-tuned on a high-quality instruction dataset can gain powerfulcapabilities to address wide-ranging code-related tasks. However, most previousexisting methods mainly view each programming language in isolation and ignorethe knowledge transfer among different programming languages. To bridge the gapamong different programming languages, we introduce a novel multi-agentcollaboration framework to enhance multilingual instruction tuning for codeLLMs, where multiple language-specific intelligent agent components withgeneration memory work together to transfer knowledge from one language toanother efficiently and effectively. Specifically, we first generate thelanguage-specific instruction data from the code snippets and then provide thegenerated data as the seed data for language-specific agents. Multiplelanguage-specific agents discuss and collaborate to formulate a new instructionand its corresponding solution (A new programming language or existingprogramming language), To further encourage the cross-lingual transfer, eachagent stores its generation history as memory and then summarizes its meritsand faults. Finally, the high-quality multilingual instruction data is used toencourage knowledge transfer among different programming languages to trainQwen2.5-xCoder. Experimental results on multilingual programming benchmarksdemonstrate the superior performance of Qwen2.5-xCoder in sharing commonknowledge, highlighting its potential to reduce the cross-lingual gap.