Large language models (LLMs) have recently transformed both the academic and
industrial landscapes due to their remarkable capacity to understand, analyze,
and generate texts based on their vast knowledge and reasoning ability.
Nevertheless, one major drawback of LLMs is their substantial computational
cost for pre-training due to their unprecedented amounts of parameters. The
disadvantage is exacerbated when new knowledge frequently needs to be
introduced into the pre-trained model. Therefore, it is imperative to develop
effective and efficient techniques to update pre-trained LLMs. Traditional
methods encode new knowledge in pre-trained LLMs through direct fine-tuning.
However, naively re-training LLMs can be computationally intensive and risks
degenerating valuable pre-trained knowledge irrelevant to the update in the
model. Recently, Knowledge-based Model Editing (KME) has attracted increasing
attention, which aims to precisely modify the LLMs to incorporate specific
knowledge, without negatively influencing other irrelevant knowledge. In this
survey, we aim to provide a comprehensive and in-depth overview of recent
advances in the field of KME. We first introduce a general formulation of KME
to encompass different KME strategies. Afterward, we provide an innovative
taxonomy of KME techniques based on how the new knowledge is introduced into
pre-trained LLMs, and investigate existing KME strategies while analyzing key
insights, advantages, and limitations of methods from each category. Moreover,
representative metrics, datasets, and applications of KME are introduced
accordingly. Finally, we provide an in-depth analysis regarding the
practicality and remaining challenges of KME and suggest promising research
directions for further advancement in this field.

介绍了大型语言模型（LLMs）的计算成本问题以及知识模型编辑（KME）领域中的最新进展和分类技术，探讨了方法的洞见、优势、限制以及实际应用和挑战，并提出了未来研究方向。