Image harmonization is a crucial technique in image composition that aims to
seamlessly match the background by adjusting the foreground of composite
images. Current methods adopt either global-level or pixel-level feature
matching. Global-level feature matching ignores the proximity prior, treating
foreground and background as separate entities. On the other hand, pixel-level
feature matching loses contextual information. Therefore, it is necessary to
use the information from semantic maps that describe different objects to guide
harmonization. In this paper, we propose Semantic-guided Region-aware Instance
Normalization (SRIN) that can utilize the semantic segmentation maps output by
a pre-trained Segment Anything Model (SAM) to guide the visual consistency
learning of foreground and background features. Abundant experiments
demonstrate the superiority of our method for image harmonization over
state-of-the-art methods.

本文提出了一种利用语义分割地图指导前景和背景特征的视觉一致性学习的方法，称为语义引导的区域感知实例标准化（SRIN），并且通过丰富的实验验证了该方法在图像协调中的优越性。

分割任意物体模型与图像融合

Segment Anything Model Meets Image Harmonization

Diffusion models (DMs) have gained prominence due to their ability to
generate high-quality, varied images, with recent advancements in text-to-image
generation. The research focus is now shifting towards the controllability of
DMs. A significant challenge within this domain is localized editing, where
specific areas of an image are modified without affecting the rest of the
content. This paper introduces LIME for localized image editing in diffusion
models that do not require user-specified regions of interest (RoI) or
additional text input. Our method employs features from pre-trained methods and
a simple clustering technique to obtain precise semantic segmentation maps.
Then, by leveraging cross-attention maps, it refines these segments for
localized edits. Finally, we propose a novel cross-attention regularization
technique that penalizes unrelated cross-attention scores in the RoI during the
denoising steps, ensuring localized edits. Our approach, without re-training
and fine-tuning, consistently improves the performance of existing methods in
various editing benchmarks.

本文介绍了一种基于扩散模型的局部图像编辑方法，利用预训练模型和简单的聚类技术获取精确的语义分割图，通过交叉注意力图对这些分割进行精细调整，并提出了一种新颖的交叉注意力正则化技术，以确保局部编辑。这种方法在各种编辑基准测试中稳定地改善了现有方法的性能。

LIME: 基于扩散模型中的关注正则化的本地图像编辑

LIME: Localized Image Editing via Attention Regularization in Diffusion  Models

Most existing methods for conditional image synthesis are only able to
generate a single plausible image for any given input, or at best a fixed
number of plausible images. In this paper, we focus on the problem of
generating images from semantic segmentation maps and present a simple new
method that can generate an arbitrary number of images with diverse appearance
for the same semantic layout. Unlike most existing approaches which adopt the
GAN framework, our method is based on the recently introduced Implicit Maximum
Likelihood Estimation (IMLE) framework. Compared to the leading approach, our
method is able to generate more diverse images while producing fewer artifacts
despite using the same architecture. The learned latent space also has sensible
structure despite the lack of supervision that encourages such behaviour.
Videos and code are available at
this https URL

本文研究了条件图像合成中的多样性问题，提出了基于 IMLE 框架的新方法，用于从语义分割映射生成具有不同外观的图像，并能够生成更多样化的图像，同时产生更少的伪影，具有合理的潜在空间结构。