Large language models (LLMs) have demonstrated remarkable performance, particularly in multilingual contexts. While recent studies suggest that LLMs can transfer skills learned in one language to others, the internal mechanisms behind this ability remain unclear. We observed that the neuron activation patterns of LLMs exhibit similarities when processing the same language, revealing the existence and location of key linguistic regions. Additionally, we found that neuron activation patterns are similar when processing sentences with the same semantic meaning in different languages. This indicates that LLMs map semantically identical inputs from different languages into a "Lingua Franca", a common semantic latent space that allows for consistent processing across languages. This semantic alignment becomes more pronounced with training and increased model size, resulting in a more language-agnostic activation pattern. Moreover, we found that key linguistic neurons are concentrated in the first and last layers of LLMs, becoming denser in the first layers as training progresses. Experiments on BLOOM and LLaMA2 support these findings, highlighting the structural evolution of multilingual LLMs during training and scaling up. This paper provides insights into the internal workings of LLMs, offering a foundation for future improvements in their cross-lingual capabilities.

本研究解决了多语言大型语言模型（LLMs）在语言能力迁移机制方面的不足，揭示出关键信息区域及其在处理相同语义内容时的神经元激活模式的相似性。研究发现，模型在训练和增大规模后会形成一个通用的语义潜在空间，从而提高跨语言处理的一致性，这一发现为今后提高大型语言模型的跨语言能力奠定了基础。

趋向通用语：多语言大型语言模型中的语言区域演变与语义对齐