Large Language Models (LLMs) have shown extraordinary capabilities in
understanding and generating text that closely mirrors human communication.
However, a primary limitation lies in the significant computational demands
during training, arising from their extensive parameterization. This challenge
is further intensified by the dynamic nature of the world, necessitating
frequent updates to LLMs to correct outdated information or integrate new
knowledge, thereby ensuring their continued relevance. Note that many
applications demand continual model adjustments post-training to address
deficiencies or undesirable behaviors. There is an increasing interest in
efficient, lightweight methods for on-the-fly model modifications. To this end,
recent years have seen a burgeoning in the techniques of knowledge editing for
LLMs, which aim to efficiently modify LLMs' behaviors within specific domains
while preserving overall performance across various inputs. In this paper, we
first define the knowledge editing problem and then provide a comprehensive
review of cutting-edge approaches. Drawing inspiration from educational and
cognitive research theories, we propose a unified categorization criterion that
classifies knowledge editing methods into three groups: resorting to external
knowledge, merging knowledge into the model, and editing intrinsic knowledge.
Furthermore, we introduce a new benchmark, KnowEdit, for a comprehensive
empirical evaluation of representative knowledge editing approaches.
Additionally, we provide an in-depth analysis of knowledge location, which can
provide a deeper understanding of the knowledge structures inherent within
LLMs. Finally, we discuss several potential applications of knowledge editing,
outlining its broad and impactful implications.

大型语言模型 (Large Language Models) 在理解和生成与人类交流非常相似的文本方面表现出非凡的能力。然而，由于其广泛的参数化，训练过程中存在重大的计算需求限制。这种挑战由于世界的动态性而进一步加剧，需要经常更新 LLM 以纠正过时的信息或整合新知识，从而确保它们持续具有相关性。近年来，对于即时修改模型的高效轻量级方法引起了越来越多的关注。本文首先定义了知识编辑问题，然后提供了对最前沿方法的全面回顾。在教育和认知研究理论的启发下，我们提出了一个统一的分类准则，将知识编辑方法分为三类：利用外部知识、将知识合并到模型中以及编辑内在知识。此外，我们引入了一个新的基准 KnowEdit，以对代表性的知识编辑方法进行综合实证评估。此外，我们对知识定位进行了深入分析，从而更深入地了解 LLM 固有的知识结构。最后，我们讨论了知识编辑的几个潜在应用，并概述了它的广泛而有影响力的意义。