Safety lies at the core of developing and deploying large language models
(LLMs). However, previous safety benchmarks only concern the safety in one
language, e.g. the majority language in the pretraining data such as English.
In this work, we build the first multilingual safety benchmark for LLMs,
XSafety, in response to the global deployment of LLMs in practice. XSafety
covers 14 kinds of commonly used safety issues across 10 languages that span
several language families. We utilize XSafety to empirically study the
multilingual safety for 4 widely-used LLMs, including both close-API and
open-source models. Experimental results show that all LLMs produce
significantly more unsafe responses for non-English queries than English ones,
indicating the necessity of developing safety alignment for non-English
languages. In addition, we propose several simple and effective prompting
methods to improve the multilingual safety of ChatGPT by evoking safety
knowledge and improving cross-lingual generalization of safety alignment. Our
prompting method can significantly reduce the ratio of unsafe responses from
19.1% to 9.7% for non-English queries. We release our data at
this https URL

我们建立了第一个多语言安全基准（XSafety）用于大型语言模型的开发和部署，在实验中发现所有的大型语言模型对非英语查询的不安全回复明显增多，为此我们提出了几种简单有效的提示方法，从而显著提高 ChatGPT 的多语言安全性。