Abstract
As large language models (LLMs) continue to advance, their capacity tofunction effectively across a diverse range of languages has shown markedimprovement. Preliminary studies observe that the hidden activations of LLMsoften resemble English, even when responding to non-English prompts. This hasled to the widespread assumption that LLMs may "think" in English. However,more recent results showing strong multilingual performance, even surpassingEnglish performance on specific tasks in other languages, challenge this view.In this work, we find that LLMs progressively develop a core language-agnosticparameter space-a remarkably small subset of parameters whose deactivationresults in significant performance degradation across all languages. Thiscompact yet critical set of parameters underlies the model's ability togeneralize beyond individual languages, supporting the emergence of abstractthought that is not tied to any specific linguistic system. Specifically, weidentify language-related neurons-those are consistently activated during theprocessing of particular languages, and categorize them as either shared(active across multiple languages) or exclusive (specific to one). As LLMsundergo continued development over time, we observe a marked increase in boththe proportion and functional importance of shared neurons, while exclusiveneurons progressively diminish in influence. These shared neurons constitutethe backbone of the core language-agnostic parameter space, supporting theemergence of abstract thought. Motivated by these insights, we proposeneuron-specific training strategies tailored to LLMs' language-agnostic levelsat different development stages. Experiments across diverse LLM familiessupport our approach.