Despite significant progress in Text-to-Image (T2I) generative models, even
lengthy and complex text descriptions still struggle to convey detailed
controls. In contrast, Layout-to-Image (L2I) generation, aiming to generate
realistic and complex scene images from user-specified layouts, has risen to
prominence. However, existing methods transform layout information into tokens
or RGB images for conditional control in the generative process, leading to
insufficient spatial and semantic controllability of individual instances. To
address these limitations, we propose a novel Spatial-Semantic Map Guided
(SSMG) diffusion model that adopts the feature map, derived from the layout, as
guidance. Owing to rich spatial and semantic information encapsulated in
well-designed feature maps, SSMG achieves superior generation quality with
sufficient spatial and semantic controllability compared to previous works.
Additionally, we propose the Relation-Sensitive Attention (RSA) and
Location-Sensitive Attention (LSA) mechanisms. The former aims to model the
relationships among multiple objects within scenes while the latter is designed
to heighten the model's sensitivity to the spatial information embedded in the
guidance. Extensive experiments demonstrate that SSMG achieves highly promising
results, setting a new state-of-the-art across a range of metrics encompassing
fidelity, diversity, and controllability.

该研究介绍了一种采用特征图作为引导的新型空间 - 语义地图引导扩散模型 (SSMG)，通过丰富的空间和语义信息，实现了优于以往工作的生成质量和充足的空间和语义可控性。实验结果表明，SSMG 在保真度、多样性和可控性等多个指标上取得了非常有前景的结果，达到了新的技术水平。