Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuning

  • 2024-11-27 18:14:38
  • Omkar Khade, Shruti Jagdale, Abhishek Phaltankar, Gauri Takalikar, Raviraj Joshi
  • 0

Abstract

Large Language Models (LLMs) have demonstrated remarkable multilingualcapabilities, yet challenges persist in adapting these models for low-resourcelanguages. In this study, we investigate the effects of Low-Rank Adaptation(LoRA) Parameter-Efficient Fine-Tuning (PEFT) on multilingual Gemma models forMarathi, a language with limited resources. Using a translated Alpaca datasetwith 52,000 instruction-response pairs, our findings reveal that whileevaluation metrics often show a performance decline post-fine-tuning, manualassessments frequently suggest that the fine-tuned models outperform theiroriginal counterparts. The observations indicate improvements in targetlanguage generation capabilities but a reduction in reasoning abilitiesfollowing language adaptation. These results underscore the need for improvedevaluation methodologies and the creation of high-quality native datasets toaccurately assess language-specific model performance in low-resource settings.

 

Quick Read (beta)

loading the full paper ...