Recent advances in large language models (LLMs) have opened up new paradigms
for accessing the knowledge stored in their parameters. One critical challenge
that has emerged is the presence of hallucinations in LLM outputs due to false
or outdated knowledge. Since retraining LLMs with updated information is
resource-intensive, there has been a growing interest in model editing.
However, many model editing methods, while effective in various scenarios, tend
to overemphasize aspects such as efficacy, generalization, and locality in
editing performance, often overlooking potential side effects on the general
abilities of LLMs. In this paper, we raise concerns that the improvement of
model factuality may come at the cost of a significant degradation of these
general abilities, which is not conducive to the sustainable development of
LLMs. Systematically, we analyze side effects by evaluating four popular
editing methods on two LLMs across eight representative task categories.
Extensive empirical research reveals that model editing does improve model
factuality but at the expense of substantially impairing general abilities.
Therefore, we advocate for more research efforts to minimize the loss of
general abilities acquired during LLM pre-training and to ultimately preserve
them during model editing.

通过对两个大型语言模型在八个代表性任务类别上评估四种常用的编辑方法进行广泛实证研究后，我们发现模型的编辑能够提高模型的事实性，但会显著损害模型的通用能力，因此我们呼吁更多的研究努力来最小化在大语言模型预训练期间获得的通用能力的损失，并在模型编辑过程中最终保留它们。