RetrieveAll: A Multilingual Named Entity Recognition Framework with Large Language Models

  • 2025-05-25 13:52:18
  • Jin Zhang, Fan Gao, Linyu Li, Yongbin Yu, Xiangxiang Wang, Nyima Tashi, Gadeng Luosang
  • 0

Abstract

The rise of large language models has led to significant performancebreakthroughs in named entity recognition (NER) for high-resource languages,yet there remains substantial room for improvement in low- and medium-resourcelanguages. Existing multilingual NER methods face severe language interferenceduring the multi-language adaptation process, manifested in feature conflictsbetween different languages and the competitive suppression of low-resourcelanguage features by high-resource languages. Although training a dedicatedmodel for each language can mitigate such interference, it lacks scalabilityand incurs excessive computational costs in real-world applications. To addressthis issue, we propose RetrieveAll, a universal multilingual NER frameworkbased on dynamic LoRA. The framework decouples task-specific features acrosslanguages and demonstrates efficient dynamic adaptability. Furthermore, weintroduce a cross-granularity knowledge augmented method that fully exploitsthe intrinsic potential of the data without relying on external resources. Byleveraging a hierarchical prompting mechanism to guide knowledge injection,this approach advances the paradigm from "prompt-guided inference" to"prompt-driven learning." Experimental results show that RetrieveAlloutperforms existing baselines; on the PAN-X dataset, it achieves an average F1improvement of 12.1 percent.

 

Quick Read (beta)

loading the full paper ...