Abstract
Translating natural languages into sign languages is a highly complex andunderexplored task. Despite growing interest in accessibility and inclusivity,the development of robust translation systems remains hindered by the limitedavailability of parallel corpora which align natural language with signlanguage data. Existing methods often struggle to generalize in thesedata-scarce environments, as the few datasets available are typicallydomain-specific, lack standardization, or fail to capture the full linguisticrichness of sign languages. To address this limitation, we propose Advanced Useof LLMs for Sign Language Translation (AulSign), a novel method that leveragesLarge Language Models via dynamic prompting and in-context learning with sampleselection and subsequent sign association. Despite their impressive abilitiesin processing text, LLMs lack intrinsic knowledge of sign languages; therefore,they are unable to natively perform this kind of translation. To overcome thislimitation, we associate the signs with compact descriptions in naturallanguage and instruct the model to use them. We evaluate our method on bothEnglish and Italian languages using SignBank+, a recognized benchmark in thefield, as well as the Italian LaCAM CNR-ISTC dataset. We demonstrate superiorperformance compared to state-of-the-art models in low-data scenario. Ourfindings demonstrate the effectiveness of AulSign, with the potential toenhance accessibility and inclusivity in communication technologies forunderrepresented linguistic communities.