Abstract
With approximately 7,000 languages spoken worldwide, current large languagemodels (LLMs) support only a small subset. Prior research indicates LLMs canlearn new languages for certain tasks without supervised data. We extend thisinvestigation to speech recognition, investigating whether LLMs can learnunseen, low-resource languages through in-context learning (ICL). Withexperiments on four diverse endangered languages that LLMs have not beentrained on, we find that providing more relevant text samples enhancesperformance in both language modelling and Automatic Speech Recognition (ASR)tasks. Furthermore, we show that the probability-based approach outperforms thetraditional instruction-based approach in language learning. Lastly, we showICL enables LLMs to achieve ASR performance that is comparable to or evensurpasses dedicated language models trained specifically for these languages,while preserving the original capabilities of the LLMs.