Abstract
Language-specific neurons in LLMs that strongly correlate with individuallanguages have been shown to influence model behavior by deactivating them.However, their role in amplification remains underexplored. This workinvestigates the effect of amplifying language-specific neurons throughinterventions across 18 languages, including low-resource ones, using threemodels primarily trained in different languages. We compare amplificationfactors by their effectiveness in steering to the target language using aproposed Language Steering Shift (LSS) evaluation score, then evaluate it ondownstream tasks: commonsense reasoning (XCOPA, XWinograd), knowledge(Include), and translation (FLORES). The optimal amplification factorseffectively steer output toward nearly all tested languages. Intervention usingthis factor on downstream tasks improves self-language performance in somecases but generally degrades cross-language results. These findings highlightthe effect of language-specific neurons in multilingual behavior, whereamplification can be beneficial especially for low-resource languages, butprovides limited advantage for cross-lingual transfer.