In recent years, large language models (LLMs) have spurred a new research
paradigm in natural language processing. Despite their excellent capability in
knowledge-based question answering and reasoning, their potential to retain
faulty or even harmful knowledge poses risks of malicious application. The
challenge of mitigating this issue and transforming these models into purer
assistants is crucial for their widespread applicability. Unfortunately,
Retraining LLMs repeatedly to eliminate undesirable knowledge is impractical
due to their immense parameters. Knowledge unlearning, derived from analogous
studies on machine unlearning, presents a promising avenue to address this
concern and is notably advantageous in the context of LLMs. It allows for the
removal of harmful knowledge in an efficient manner, without affecting
unrelated knowledge in the model. To this end, we provide a survey of knowledge
unlearning in the era of LLMs. Firstly, we formally define the knowledge
unlearning problem and distinguish it from related works. Subsequently, we
categorize existing knowledge unlearning methods into three classes: those
based on parameter optimization, parameter merging, and in-context learning,
and introduce details of these unlearning methods. We further present
evaluation datasets used in existing methods, and finally conclude this survey
by presenting the ongoing challenges and future directions.

近年来，大型语言模型（LLM）在自然语言处理领域引起了一种新的研究范式。然而，这些模型保留错误或甚至有害知识的潜力带来了恶意应用的风险。为了使这些模型能够广泛应用，解决这个问题并将其转化为纯粹的助手是至关重要的。为此，我们提供了一份关于 LLM 时代的知识遗忘的综述。我们从正式定义知识遗忘问题并与相关工作区分开始。随后，我们将现有的知识遗忘方法分为三类：基于参数优化、参数合并和上下文学习的方法，并介绍这些方法的细节。我们还介绍了现有方法中使用的评估数据集，并最后总结了这份综述中存在的挑战和未来的方向。