Abstract
In this paper, we propose SignLLM, a multilingual Sign Language Production(SLP) large language model, which includes two novel multilingual SLP modesMLSF and Prompt2LangGloss that allow sign language gestures generation fromquery texts input and question-style prompts input respectively. Both modes canuse a new RL loss based on reinforcement learning and a new RL module namedPriority Learning Channel. These RL components can accelerate the training byenhancing the model's capability to sample high-quality data. To train SignLLM,we introduce Prompt2Sign, a comprehensive multilingual sign language dataset,which builds from public data, including American Sign Language (ASL) and sevenothers. This dataset standardizes information by extracting pose informationfrom sign language videos into a unified compressed format. We extensivelyevaluate SignLLM, demonstrating that our model achieves state-of-the-artperformance on SLP tasks across eight sign languages.