Large language models (LLMs) store extensive factual knowledge, but the
mechanisms behind how they store and express this knowledge remain unclear. The
Knowledge Neuron (KN) thesis is a prominent theory for explaining these
mechanisms. This theory is based on the knowledge localization (KL) assumption,
which suggests that a fact can be localized to a few knowledge storage units,
namely knowledge neurons. However, this assumption may be overly strong
regarding knowledge storage and neglects knowledge expression mechanisms. Thus,
we re-examine the KL assumption and confirm the existence of facts that do not
adhere to it from both statistical and knowledge modification perspectives.
Furthermore, we propose the Query Localization (QL) assumption. (1) Query-KN
Mapping: The localization results are associated with the query rather than the
fact. (2) Dynamic KN Selection: The attention module contributes to the
selection of KNs for answering a query. Based on this, we further propose the
Consistency-Aware KN modification method, which improves the performance of
knowledge modification. We conduct 39 sets of experiments, along with
additional visualization experiments, to rigorously validate our conclusions.

大型语言模型可以存储广泛的事实知识，但其存储和表达这些知识的机制仍不清楚。本论文重新考察了知识定位假设的合理性，并从统计学和知识修改的角度确认了不符合该假设的事实的存在。进一步提出了关联查询和知识选择的方法，以及一种改进知识修改的一致性感知方法。通过 39 组实验证实了我们的结论。

知识本地化：任务未完成？进入查询本地化！

Knowledge Localization: Mission Not Accomplished? Enter Query  Localization!

Large Transformer models have achieved impressive performance in many natural
language tasks. In particular, Transformer based language models have been
shown to have great capabilities in encoding factual knowledge in their vast
amount of parameters. While the tasks of improving the memorization and
generalization of Transformers have been widely studied, it is not well known
how to make transformers forget specific old facts and memorize new ones. In
this paper, we propose a new task of \emph{explicitly modifying specific
factual knowledge in Transformer models while ensuring the model performance
does not degrade on the unmodified facts}. This task is useful in many
scenarios, such as updating stale knowledge, protecting privacy, and
eliminating unintended biases stored in the models. We benchmarked several
approaches that provide natural baseline performances on this task. This leads
to the discovery of key components of a Transformer model that are especially
effective for knowledge modifications. The work also provides insights into the
role that different training phases (such as pretraining and fine-tuning) play
towards memorization and knowledge modification.

本文提出了一个新任务，即在确保模型在未修改的信息方面的性能不降低的情况下显式修改 Transformer 模型中特定的事实知识，并在此任务上基准化了几种方法，发现了用于知识修改的 Transformer 模型的关键组件，并提供了关于不同训练阶段（如预训练和微调）对记忆和知识修改所起作用的见解。