Multilingual large language models (LLMs) have greatly increased the ceiling
of performance on non-English tasks. However the mechanisms behind
multilingualism in these LLMs are poorly understood. Of particular interest is
the degree to which internal representations are shared between languages.
Recent work on neuron analysis of LLMs has focused on the monolingual case, and
the limited work on the multilingual case has not considered the interaction
between tasks and linguistic representations. In our work, we investigate how
neuron activation is shared across languages by categorizing neurons into four
distinct groups according to their responses across different languages for a
particular input: all-shared, partial-shared, specific, and non-activated. This
categorization is combined with a study of neuron attribution, i.e. the
importance of a neuron w.r.t an output. Our analysis reveals the following
insights: (i) the linguistic sharing patterns are strongly affected by the type
of task, but neuron behaviour changes across different inputs even for the same
task; (ii) all-shared neurons play a key role in generating correct responses;
(iii) boosting multilingual alignment by increasing all-shared neurons can
enhance accuracy on multilingual tasks. The code is available at
this https URL

多语言大型语言模型（LLMs）的多语言机制对其内部表示的共享程度进行研究，并发现语言分享模式受任务类型和输入的影响，所有共享神经元在生成正确响应中起关键作用，增加所有共享神经元可以提高多语言任务的准确性。