Layout-to-image synthesis is an emerging technique in conditional image
generation. It aims to generate complex scenes, where users require fine
control over the layout of the objects in a scene. However, it remains
challenging to control the object coherence, including semantic coherence
(e.g., the cat looks at the flowers or not) and physical coherence (e.g., the
hand and the racket should not be misaligned). In this paper, we propose a
novel diffusion model with effective global semantic fusion (GSF) and
self-similarity feature enhancement modules to guide the object coherence for
this task. For semantic coherence, we argue that the image caption contains
rich information for defining the semantic relationship within the objects in
the images. Instead of simply employing cross-attention between captions and
generated images, which addresses the highly relevant layout restriction and
semantic coherence separately and thus leads to unsatisfying results shown in
our experiments, we develop GSF to fuse the supervision from the layout
restriction and semantic coherence requirement and exploit it to guide the
image synthesis process. Moreover, to improve the physical coherence, we
develop a Self-similarity Coherence Attention (SCA) module to explicitly
integrate local contextual physical coherence into each pixel's generation
process. Specifically, we adopt a self-similarity map to encode the coherence
restrictions and employ it to extract coherent features from text embedding.
Through visualization of our self-similarity map, we explore the essence of
SCA, revealing that its effectiveness is not only in capturing reliable
physical coherence patterns but also in enhancing complex texture generation.
Extensive experiments demonstrate the superiority of our proposed method in
both image generation quality and controllability.

我们提出了一种新颖的扩散模型，通过全局语义融合和自相似特征增强模块来引导对象的一致性，并采用自相似性一致性注意模块来显式地将局部背景一致性整合到每个像素的生成过程中。实验结果表明，该方法在图像生成质量和可控性方面具有优势。