Abstract
Large Language Models (LLMs) have shown remarkable capabilities in naturallanguage processing but exhibit significant performance gaps among differentlanguages. Most existing approaches to address these disparities rely onpretraining or fine-tuning, which are resource-intensive. To overcome theselimitations without incurring significant costs, we propose Inference-TimeCross-Lingual Intervention (INCLINE), a novel framework that enhances LLMperformance on low-performing (source) languages by aligning their internalrepresentations with those of high-performing (target) languages duringinference. INCLINE initially learns alignment matrices using parallel sentencesfrom source and target languages through a Least-Squares optimization, and thenapplies these matrices during inference to transform the low-performinglanguage representations toward the high-performing language space. Extensiveexperiments on nine benchmarks with five LLMs demonstrate that INCLINEsignificantly improves performance across diverse tasks and languages, comparedto recent strong baselines. Our analysis demonstrates that INCLINE is highlycost-effective and applicable to a wide range of applications. In addition, werelease the code to foster research along this line:https://github.com/weixuan-wang123/INCLINE.