Large language models (LLMs) demonstrate remarkable multilingual capabilities
without being pre-trained on specially curated multilingual parallel corpora.
It remains a challenging problem to explain the underlying mechanisms by which
LLMs process multilingual texts. In this paper, we delve into the composition
of Transformer architectures in LLMs to pinpoint language-specific regions.
Specially, we propose a novel detection method, language activation probability
entropy (LAPE), to identify language-specific neurons within LLMs. Based on
LAPE, we conduct comprehensive experiments on two representative LLMs, namely
LLaMA-2 and BLOOM. Our findings indicate that LLMs' proficiency in processing a
particular language is predominantly due to a small subset of neurons,
primarily situated in the models' top and bottom layers. Furthermore, we
showcase the feasibility to "steer" the output language of LLMs by selectively
activating or deactivating language-specific neurons. Our research provides
important evidence to the understanding and exploration of the multilingual
capabilities of LLMs.

通过新的检测方法 - 语言激活概率熵（LAPE），我们研究了大型语言模型中的 Transformer 架构，以确定语言特定的区域，并显示了激活或关闭特定语言神经元对大型语言模型输出语言的可控性。