Performance of Recent Large Language Models for a Low-Resourced Language

  • 2024-07-31 05:38:07
  • Ravindu Jayakody, Gihan Dias
  • 0

Abstract

Large Language Models (LLMs) have shown significant advances in the pastyear. In addition to new versions of GPT and Llama, several other LLMs havebeen introduced recently. Some of these are open models available for downloadand modification. Although multilingual large language models have been available for sometime, their performance on low-resourced languages such as Sinhala has beenpoor. We evaluated four recent LLMs on their performance directly in theSinhala language, and by translation to and from English. We also evaluatedtheir fine-tunability with a small amount of fine-tuning data. Claude and GPT4o perform well out-of-the-box and do significantly better than previousversions. Llama and Mistral perform poorly but show some promise of improvementwith fine tuning.

 

Quick Read (beta)

loading the full paper ...