Identifying layers within text-to-image models which control visual
attributes can facilitate efficient model editing through closed-form updates.
Recent work, leveraging causal tracing show that early Stable-Diffusion
variants confine knowledge primarily to the first layer of the CLIP
text-encoder, while it diffuses throughout the UNet.Extending this framework,
we observe that for recent models (e.g., SD-XL, DeepFloyd), causal tracing
fails in pinpointing localized knowledge, highlighting challenges in model
editing. To address this issue, we introduce the concept of Mechanistic
Localization in text-to-image models, where knowledge about various visual
attributes (e.g., ``style", ``objects", ``facts") can be mechanistically
localized to a small fraction of layers in the UNet, thus facilitating
efficient model editing. We localize knowledge using our method LocoGen which
measures the direct effect of intermediate layers to output generation by
performing interventions in the cross-attention layers of the UNet. We then
employ LocoEdit, a fast closed-form editing method across popular open-source
text-to-image models (including the latest SD-XL)and explore the possibilities
of neuron-level model editing. Using Mechanistic Localization, our work offers
a better view of successes and failures in localization-based text-to-image
model editing. Code will be available at
\href{https://github.com/samyadeepbasu/LocoGen}{this https URL}.

通过定位知识，可以在文本到图像模型中实现更高效的模型编辑，提供了对基于定位的文本到图像模型编辑中成功和失败的更好视角。

文本到图像生成模型中的机制性知识定位

On Mechanistic Knowledge Localization in Text-to-Image Generative Models

Large Language Models are successfully adopted in software engineering,
especially in code generation. Updating these models with new knowledge is very
expensive, and is often required to fully realize their value. In this paper,
we propose a novel and effective model editing approach, \textsc{MENT}, to
patch LLMs in coding tasks. Based on the mechanism of generative LLMs,
\textsc{MENT} enables model editing in next-token predictions, and further
supports common coding tasks. \textsc{MENT} is effective, efficient, and
reliable. It can correct a neural model by patching 1 or 2 neurons. As the
pioneer work on neuron-level model editing of generative models, we formalize
the editing process and introduce the involved concepts. Besides, we also
introduce new measures to evaluate its generalization ability, and build a
benchmark for further study. Our approach is evaluated on three coding tasks,
including API-seq recommendation, line-level code generation, and
pseudocode-to-code transaction. It outperforms the state-of-the-art by a
significant margin on both effectiveness and efficiency measures. In addition,
we demonstrate the usages of \textsc{MENT} for LLM reasoning in software
engineering. By editing the LLM knowledge with \textsc{MENT}, the directly or
indirectly dependent behaviors in the chain-of-thought change accordingly and
automatically.

大型语言模型在软件工程领域中成功应用，特别是在代码生成方面。本文提出了一种新颖有效的模型编辑方法（	extsc {MENT}），用于修补编程任务中的 LLMs。 	extsc {MENT} 基于生成式 LLMs 的机制，在下一个标记预测中实现模型编辑，并进一步支持常见的编码任务。本研究提出了形式化的编辑过程和相关概念，并引入了评估其泛化能力的新指标。在 API-seq 推荐、行级代码生成和伪代码转换任务上，我们的方法优于最先进的方法，并展示了	extsc {MENT} 在软件工程中的应用。