Large Language Models (LLMs) can become outdated over time as they may lack
updated world knowledge, leading to factual knowledge errors and gaps.
Knowledge Editing (KE) aims to overcome this challenge using weight updates
that do not require expensive retraining. We propose treating KE as an LLM
alignment problem. Toward this goal, we introduce Knowledge Direct Preference
Optimization (KDPO), a variation of the Direct Preference Optimization (DPO)
that is more effective for knowledge modifications. Our method is based on an
online approach that continually updates the knowledge stored in the model. We
use the current knowledge as a negative sample and the new knowledge we want to
introduce as a positive sample in a process called DPO. We also use
teacher-forcing for negative sample generation and optimize using the positive
sample, which helps maintain localized changes. We tested our KE method on
various datasets and models, comparing it to several cutting-edge methods, with
100 and 500 sequential edits. Additionally, we conducted an ablation study
comparing our method to the standard DPO approach. Our experimental results
show that our modified DPO method allows for more refined KE, achieving similar
or better performance compared to previous methods.

基于在线方法的知识编辑，使用当前已知知识作为负样本，并引入新知识作为正样本，通过使用改进的 DPO 方法，进一步优化知识编辑，以实现与先前方法相似或更好的性能。