Abstract
Multilingual large language models (LLMs) open up new possibilities forleveraging information across languages, but their factual knowledge recallremains inconsistent depending on the input language. While previous studieshave attempted to address this issue through English-based prompting andevaluation, we explore non-English to English transfer via Language and ThoughtTheory. This perspective allows us to examine language-thought binding in LLMsand uncover why factual knowledge often fails to transfer effectively. Wepropose the Language-to-Thought (L2T) prompting strategy, which analyzes therelationship between input language, internal cognitive processes, andknowledge. Experimental results challenge the assumption that English-basedapproaches consistently outperform other languages and offer a novel insightthat aligning the model's internal thought with the knowledge required for thetask is critical for successful cross-lingual transfer. Furthermore, we showthat applying L2T during training can alleviate LLMs' reliance on the inputlanguage and facilitate cross-linguistic knowledge integration withouttranslation-based learning. Code and datasets will be available.