In the field of language model interpretability, circuit discovery is gaining
popularity. Despite this, the true meaning of these circuits remain largely
unanswered. We introduce a novel method to learn their meanings as a holistic
object through the lens of knowledge editing. We extract circuits in the
GPT2-XL model using diverse text classification datasets, and use hierarchical
relations datasets to explore knowledge editing in the circuits. Our findings
indicate that these circuits contain entity knowledge but resist new knowledge
more than complementary circuits during knowledge editing. Additionally, we
examine the impact of circuit size, discovering that an ideal "theoretical
circuit" where essential knowledge is concentrated likely incorporates more
than 5% but less than 50% of the model's parameters. We also assess the overlap
between circuits from different datasets, finding moderate similarities. What
constitutes these circuits, then? We find that up to 60% of the circuits
consist of layer normalization modules rather than attention or MLP modules,
adding evidence to the ongoing debates regarding knowledge localization. In
summary, our findings offer new insights into the functions of the circuits,
and introduce research directions for further interpretability and safety
research of language models.

通过知识编辑的视角，我们介绍了一种学习语言模型电路的新方法。通过在 GPT2-XL 模型中提取电路，使用多样的文本分类数据集和层次关系数据集，我们发现这些电路包含实体知识，但在知识编辑中对新知识的抵抗力高于互补电路。此外，我们研究了电路尺寸的影响，发现理想的电路可能包含模型参数的 5% 至 50%。我们还检查了不同数据集中电路之间的重叠情况，发现中等相似性。总之，我们的研究结果为电路的功能提供了新的见解，并引入了进一步解释性和安全性研究的方向。

电路意味着什么？知识编辑视角

What Do the Circuits Mean? A Knowledge Edit View

Large language models (LLMs) store extensive factual knowledge, but the
mechanisms behind how they store and express this knowledge remain unclear. The
Knowledge Neuron (KN) thesis is a prominent theory for explaining these
mechanisms. This theory is based on the knowledge localization (KL) assumption,
which suggests that a fact can be localized to a few knowledge storage units,
namely knowledge neurons. However, this assumption may be overly strong
regarding knowledge storage and neglects knowledge expression mechanisms. Thus,
we re-examine the KL assumption and confirm the existence of facts that do not
adhere to it from both statistical and knowledge modification perspectives.
Furthermore, we propose the Query Localization (QL) assumption. (1) Query-KN
Mapping: The localization results are associated with the query rather than the
fact. (2) Dynamic KN Selection: The attention module contributes to the
selection of KNs for answering a query. Based on this, we further propose the
Consistency-Aware KN modification method, which improves the performance of
knowledge modification. We conduct 39 sets of experiments, along with
additional visualization experiments, to rigorously validate our conclusions.

大型语言模型可以存储广泛的事实知识，但其存储和表达这些知识的机制仍不清楚。本论文重新考察了知识定位假设的合理性，并从统计学和知识修改的角度确认了不符合该假设的事实的存在。进一步提出了关联查询和知识选择的方法，以及一种改进知识修改的一致性感知方法。通过 39 组实验证实了我们的结论。

知识本地化：任务未完成？进入查询本地化！

Knowledge Localization: Mission Not Accomplished? Enter Query  Localization!

Identifying layers within text-to-image models which control visual
attributes can facilitate efficient model editing through closed-form updates.
Recent work, leveraging causal tracing show that early Stable-Diffusion
variants confine knowledge primarily to the first layer of the CLIP
text-encoder, while it diffuses throughout the UNet.Extending this framework,
we observe that for recent models (e.g., SD-XL, DeepFloyd), causal tracing
fails in pinpointing localized knowledge, highlighting challenges in model
editing. To address this issue, we introduce the concept of Mechanistic
Localization in text-to-image models, where knowledge about various visual
attributes (e.g., ``style", ``objects", ``facts") can be mechanistically
localized to a small fraction of layers in the UNet, thus facilitating
efficient model editing. We localize knowledge using our method LocoGen which
measures the direct effect of intermediate layers to output generation by
performing interventions in the cross-attention layers of the UNet. We then
employ LocoEdit, a fast closed-form editing method across popular open-source
text-to-image models (including the latest SD-XL)and explore the possibilities
of neuron-level model editing. Using Mechanistic Localization, our work offers
a better view of successes and failures in localization-based text-to-image
model editing. Code will be available at
\href{https://github.com/samyadeepbasu/LocoGen}{this https URL}.

通过定位知识，可以在文本到图像模型中实现更高效的模型编辑，提供了对基于定位的文本到图像模型编辑中成功和失败的更好视角。