Recent knowledge editing methods have primarily focused on modifying
structured knowledge in large language models, heavily relying on the
assumption that structured knowledge is stored as key-value pairs locally in
MLP layers or specific neurons. However, this task setting overlooks the fact
that a significant portion of real-world knowledge is stored in an unstructured
format, characterized by long-form content, noise, and a complex yet
comprehensive nature. The "knowledge locating" and "term-driven optimization"
techniques conducted from the assumption used in previous methods (e.g., MEMIT)
are ill-suited for unstructured knowledge. To address these challenges, we
propose a novel unstructured knowledge editing method, namely UnKE, which
extends previous assumptions in the layer dimension and token dimension.
Firstly, in the layer dimension, we discard the "knowledge locating" step and
treat first few layers as the key, which expand knowledge storage through
layers to break the "knowledge stored locally" assumption. Next, we replace
"term-driven optimization" with "cause-driven optimization" across all inputted
tokens in the token dimension, directly optimizing the last layer of the key
generator to perform editing to generate the required key vectors. By utilizing
key-value pairs at the layer level, UnKE effectively represents and edits
complex and comprehensive unstructured knowledge, leveraging the potential of
both the MLP and attention layers. Results on newly proposed unstructure
knowledge editing dataset (UnKEBench) and traditional structured datasets
demonstrate that UnKE achieves remarkable performance, surpassing strong
baselines.

提出了一种名为 UnKE 的新型非结构化知识编辑方法，通过在层次和标记维度进行扩展，实现了对复杂和全面的非结构化知识的有效表示和编辑，取得了显著的性能提升。