Abstract
The Fon language, spoken by an average 2 million of people, is a trulylow-resourced African language, with a limited online presence, and existingdatasets (just to name but a few). Multitask learning is a learning paradigmthat aims to improve the generalization capacity of a model by sharingknowledge across different but related tasks: this could be prevalent in verydata-scarce scenarios. In this paper, we present the first explorative approachto multitask learning, for model capabilities enhancement in Natural LanguageProcessing for the Fon language. Specifically, we explore the tasks of NamedEntity Recognition (NER) and Part of Speech Tagging (POS) for Fon. We leveragetwo language model heads as encoders to build shared representations for theinputs, and we use linear layers blocks for classification relative to eachtask. Our results on the NER and POS tasks for Fon, show competitive (orbetter) performances compared to several multilingual pretrained languagemodels finetuned on single tasks. Additionally, we perform a few ablationstudies to leverage the efficiency of two different loss combination strategiesand find out that the equal loss weighting approach works best in our case. Ourcode is open-sourced at https://github.com/bonaventuredossou/multitask_fon.