Large-scale Text-to-Image (T2I) diffusion models demonstrate significant
generation capabilities based on textual prompts. Based on the T2I diffusion
models, text-guided image editing research aims to empower users to manipulate
generated images by altering the text prompts. However, existing image editing
techniques are prone to editing over unintentional regions that are beyond the
intended target area, primarily due to inaccuracies in cross-attention maps. To
address this problem, we propose Localization-aware Inversion (LocInv), which
exploits segmentation maps or bounding boxes as extra localization priors to
refine the cross-attention maps in the denoising phases of the diffusion
process. Through the dynamic updating of tokens corresponding to noun words in
the textual input, we are compelling the cross-attention maps to closely align
with the correct noun and adjective words in the text prompt. Based on this
technique, we achieve fine-grained image editing over particular objects while
preventing undesired changes to other regions. Our method LocInv, based on the
publicly available Stable Diffusion, is extensively evaluated on a subset of
the COCO dataset, and consistently obtains superior results both quantitatively
and qualitatively.The code will be released at
this https URL

基于 T2I 扩散模型，本研究提出了一种局部感知反演（LocInv）方法，通过利用分割地图或边界框作为额外的定位先验，精确修正扩散过程中的交互关注图，使其与文本提示中的正确名词和形容词单词紧密对齐，从而实现对特定对象的细粒度图像编辑，并防止对其他区域产生不希望的改变。该方法在 COCO 数据集的子集上经过广泛评估，定量和定性地都取得了优秀的结果。