Abstract
High-resource language models often fall short in the African context, wherethere is a critical need for models that are efficient, accessible, and locallyrelevant, even amidst significant computing and data constraints. This paperintroduces InkubaLM, a small language model with 0.4 billion parameters, whichachieves performance comparable to models with significantly larger parametercounts and more extensive training data on tasks such as machine translation,question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLMoutperforms many larger models in sentiment analysis and demonstratesremarkable consistency across multiple languages. This work represents apivotal advancement in challenging the conventional paradigm that effectivelanguage models must rely on substantial resources. Our model and datasets arepublicly available \footnote{\url{https://huggingface.co/lelapa}} to encourageresearch and development on low-resource languages.