Abstract
Large Language Models have demonstrated impressive reasoning capabilitiesacross multiple languages. However, the relationship between capabilities indifferent languages is less explored. In this work, we decompose the process ofreasoning tasks into two separated components: knowledge retrieval andknowledge-free reasoning, and analyze the relationship between cross-lingualtransferability and these two components. With adapted commonsense reasoningdatasets and constructed knowledge-free reasoning datasets, we show that theknowledge-free reasoning capability can be nearly perfectly transferred acrossvarious source-target language directions despite the secondary impact ofresource in some specific target languages, while cross-lingual knowledgeretrieval significantly hinders the transfer. Moreover, by analyzing the hiddenstates and feed-forward network neuron activation during the reasoning, we showthat higher similarity of hidden representations and larger overlap ofactivated neurons could explain the better cross-lingual transferability ofknowledge-free reasoning than knowledge retrieval. Thus, we hypothesize thatknowledge-free reasoning shares similar neurons in different languages forreasoning, while knowledge is stored separately in different languages. Ourcode and data is available at:https://github.com/NJUNLP/Knowledge-Free-Reasoning.