By training on text in various languages, large language models (LLMs)
typically possess multilingual support and demonstrate remarkable capabilities
in solving tasks described in different languages. However, LLMs can exhibit
linguistic discrimination due to the uneven distribution of training data
across languages. That is, LLMs are hard to keep the consistency of responses
when faced with the same task but depicted in different languages.
In this study, we first explore the consistency in the LLMs' outputs
responding to queries in various languages from two aspects: safety and
quality. We conduct this analysis with two datasets (AdvBench and NQ) based on
four LLMs (Llama2-13b, Gemma-7b, GPT-3.5-turbo and Gemini-pro). The results
show that LLMs exhibit stronger human alignment capabilities with queries in
English, French, Russian, and Spanish (only 1.04\% of harmful queries
successfully jailbreak on average) compared to queries in Bengali, Georgian,
Nepali and Maithili (27.7\% of harmful queries jailbreak successfully on
average). Moreover, for queries in English, Danish, Czech and Slovenian, LLMs
tend to produce responses with a higher quality (with 0.1494 $F_1$ score on
average) compared to the other languages. Upon these findings, we propose
LDFighter, a similarity-based voting, to mitigate the linguistic discrimination
in LLMs. LDFighter ensures consistent service for different language speakers.
We evaluate LDFighter with both benign queries and harmful queries. The results
show that LDFighter not only significantly reduces the jailbreak success rate
but also improve the response quality on average, demonstrating its
effectiveness.

通过使用多种语言进行训练，大型语言模型（LLMs）通常具有多语言支持，并展示出在不同语言描述的任务中解决问题的显著能力。然而，由于在不同语言之间训练数据的不均衡分布，LLMs 可能表现出语言歧视，即面对相同任务但在不同语言中描述时，难以保持一致的响应。在本研究中，我们首先从安全性和质量两个方面探讨 LLMs 在对各种语言的查询产生的输出的一致性。我们使用两个数据集（AdvBench 和 NQ）对四个 LLMs（Llama2-13b、Gemma-7b、GPT-3.5-turbo 和 Gemini-pro）进行分析。结果显示，与孟加拉语、格鲁吉亚语、尼泊尔语和迈蒂利语的查询相比，LLMs 对英语、法语、俄语和西班牙语的查询表现出更强的人类对齐能力（平均有 1.04％的有害查询成功越狱）。此外，对于英语、丹麦语、捷克语和斯洛文尼亚语的查询，LLMs 倾向于产生具有更高质量的响应（平均 F1 得分为 0.1494），与其他语言相比。基于这些发现，我们提出了一种基于相似度的投票机制 LDFighter 来减轻 LLMs 中的语言歧视。LDFighter 确保为不同语言使用者提供一致的服务。我们使用良性查询和有害查询来评估 LDFighter。结果显示，LDFighter 不仅显著降低了成功越狱的概率，还平均改善了响应质量，证明了其有效性。