Recent text-to-image (T2I) diffusion models have achieved remarkable progress
in generating high-quality images given text-prompts as input. However, these
models fail to convey appropriate spatial composition specified by a layout
instruction. In this work, we probe into zero-shot grounded T2I generation with
diffusion models, that is, generating images corresponding to the input layout
information without training auxiliary modules or finetuning diffusion models.
We propose a Region and Boundary (R&B) aware cross-attention guidance approach
that gradually modulates the attention maps of diffusion model during
generative process, and assists the model to synthesize images (1) with high
fidelity, (2) highly compatible with textual input, and (3) interpreting layout
instructions accurately. Specifically, we leverage the discrete sampling to
bridge the gap between consecutive attention maps and discrete layout
constraints, and design a region-aware loss to refine the generative layout
during diffusion process. We further propose a boundary-aware loss to
strengthen object discriminability within the corresponding regions.
Experimental results show that our method outperforms existing state-of-the-art
zero-shot grounded T2I generation methods by a large margin both qualitatively
and quantitatively on several benchmarks.

这篇论文研究了利用扩散模型进行零样本生成的文本到图像转换方法，提出了一种基于区域和边界感知的交叉注意力引导方法，通过渐进调节扩散模型的注意力图来生成与输入布局信息相对应的高保真度、与文本输入高度兼容、准确解释布局指令的图像。实验证明，该方法在多个基准测试上在质量和数量上均优于现有的零样本生成方法。