Diffusion models (DMs) have gained prominence due to their ability to
generate high-quality, varied images, with recent advancements in text-to-image
generation. The research focus is now shifting towards the controllability of
DMs. A significant challenge within this domain is localized editing, where
specific areas of an image are modified without affecting the rest of the
content. This paper introduces LIME for localized image editing in diffusion
models that do not require user-specified regions of interest (RoI) or
additional text input. Our method employs features from pre-trained methods and
a simple clustering technique to obtain precise semantic segmentation maps.
Then, by leveraging cross-attention maps, it refines these segments for
localized edits. Finally, we propose a novel cross-attention regularization
technique that penalizes unrelated cross-attention scores in the RoI during the
denoising steps, ensuring localized edits. Our approach, without re-training
and fine-tuning, consistently improves the performance of existing methods in
various editing benchmarks.

本文介绍了一种基于扩散模型的局部图像编辑方法，利用预训练模型和简单的聚类技术获取精确的语义分割图，通过交叉注意力图对这些分割进行精细调整，并提出了一种新颖的交叉注意力正则化技术，以确保局部编辑。这种方法在各种编辑基准测试中稳定地改善了现有方法的性能。