The mechanisms behind multilingual capabilities in Large Language Models (LLMs) have been examined using neuron-based or internal-activation-based methods. However, these methods often face challenges such as superposition and layer-wise activation variance, which limit their reliability. Sparse Autoencoders (SAEs) offer a more nuanced analysis by decomposing the activations of LLMs into sparse linear combination of SAE features. We introduce a novel metric to assess the monolinguality of features obtained from SAEs, discovering that some features are strongly related to specific languages. Additionally, we show that ablating these SAE features only significantly reduces abilities in one language of LLMs, leaving others almost unaffected. Interestingly, we find some languages have multiple synergistic SAE features, and ablating them together yields greater improvement than ablating individually. Moreover, we leverage these SAE-derived language-specific features to enhance steering vectors, achieving control over the language generated by LLMs.

本研究针对大型语言模型在多语言能力方面的机制，提出了使用稀疏自编码器（SAE）作为更可靠的分析工具，从而克服传统方法的局限性。研究发现，从SAE获得的特征与特定语言密切相关，通过选择性去除这些特征，能够显著改善大型语言模型的语言控制能力。

通过稀疏自编码器揭示大型语言模型中的语言特征