This paper investigates the propagation of harmful information in
multilingual large language models (LLMs) and evaluates the efficacy of various
unlearning methods. We demonstrate that fake information, regardless of the
language it is in, once introduced into these models through training data, can
spread across different languages, compromising the integrity and reliability
of the generated content. Our findings reveal that standard unlearning
techniques, which typically focus on English data, are insufficient in
mitigating the spread of harmful content in multilingual contexts and could
inadvertently reinforce harmful content across languages. We show that only by
addressing harmful responses in both English and the original language of the
harmful data can we effectively eliminate generations for all languages. This
underscores the critical need for comprehensive unlearning strategies that
consider the multilingual nature of modern LLMs to enhance their safety and
reliability across diverse linguistic landscapes.

研究探究了有害信息在多语种大型语言模型中的传播，并评估了各种反学习方法的有效性。我们证明了无论以哪种语言出现，一旦通过训练数据引入虚假信息到这些模型中，它可以在不同语言之间传播，从而损害生成内容的完整性和可靠性。我们的发现揭示了标准反学习技术通常只关注英文数据，无法有效减轻多语种环境中有害内容的传播，并可能无意中在各种语言之间加强有害内容。我们展示了只有同时解决有害数据的英文和原始语言中的有害回应，我们才能有效地消除所有语言的生成。这强调了考虑现代多语种大型语言模型的多语言特性来增强其安全性和可靠性的全面反学习策略的重要性。