Recent studies have demonstrated that large language models (LLMs) store
massive factual knowledge within their parameters. But existing LLMs are prone
to hallucinate unintended text due to false or outdated knowledge. Since
retraining LLMs is resource intensive, there has been a growing interest in the
concept of model editing. Despite the emergence of benchmarks and approaches,
these unidirectional editing and evaluation have failed to explore the reversal
curse. Intuitively, if "The capital of France is" is edited to be a counterfact
"London" within a model, then it should be able to naturally reason and recall
the reverse fact, i.e., "London is the capital of" followed by "France" instead
of "England". In this paper, we study bidirectional language model editing,
aiming to provide rigorous model editing evaluation to assess if edited LLMs
can recall the editing knowledge bidirectionally. A new evaluation metric of
reversibility is introduced, and a benchmark dubbed as Bidirectional Assessment
for Knowledge Editing (BAKE) is constructed to evaluate the reversibility of
edited models in recalling knowledge in the reverse direction of editing. We
surprisingly observe that while current editing methods and LLMs can
effectively recall editing facts in the direction of editing, they suffer
serious deficiencies when evaluated in the reverse direction. To mitigate the
reversal curse, a method named Bidirectionally Inversible Relationship moDeling
(BIRD) is proposed. A set of editing objectives that incorporate bidirectional
relationships between subject and object into the updated model weights are
designed. Experiments show that BIRD improves the performance of four
representative LLMs of different sizes via question answering and judgement.

本文研究双向语言模型编辑，旨在提供严格的模型编辑评估，以评估编辑后的语言模型是否能双向回忆编辑的知识。引入了一种新的可逆性评估度量，并构建了一个名为 Bidirectional Assessment for Knowledge Editing (BAKE) 的基准来评估编辑后模型在回忆编辑的知识反向方向上的可逆性。我们惊奇地发现，虽然当前的编辑方法和语言模型在编辑方向上能有效地回忆编辑的事实，但在反向方向上评估时存在严重不足。为了缓解这种逆转的问题，提出了一种名为 Bidirectionally Inversible Relationship moDeling (BIRD) 的方法。设计了一组编辑目标，将主体和客体之间的双向关系纳入更新后的模型权重中。实验证明，BIRD 通过问答和判断提高了四个不同规模的代表性语言模型的性能。