Abstract
In this paper, we propose and describe the first open Llama2 large languagemodels (LLMs) for the Lithuanian language, including an accompanyingquestion/answer (Q/A) dataset and translations of popular LLM benchmarks. Weprovide a brief review of open regional LLMs and detailed information on theproposed LLMs and their training process. We also conduct an empiricalevaluation, comparing the perplexities of the proposed LLMs with those of othermodern open LLMs. In addition, benchmarking the proposed LLMs against languageunderstanding tasks reveals that high-quality pretraining datasets may beessential for achieving models that perform efficiently on these benchmarks.The full realisations of the described LLMs are available in the accompanyingopen repository~\url{https://huggingface.co/neurotechnology}.