Text-to-image synthesis has achieved high-quality results with recent
advances in diffusion models. However, text input alone has high spatial
ambiguity and limited user controllability. Most existing methods allow spatial
control through additional visual guidance (e.g, sketches and semantic masks)
but require additional training with annotated images. In this paper, we
propose a method for spatially controlling text-to-image generation without
further training of diffusion models. Our method is based on the insight that
the cross-attention maps reflect the positional relationship between words and
pixels. Our aim is to control the attention maps according to given semantic
masks and text prompts. To this end, we first explore a simple approach of
directly swapping the cross-attention maps with constant maps computed from the
semantic regions. Moreover, we propose masked-attention guidance, which can
generate images more faithful to semantic masks than the first approach.
Masked-attention guidance indirectly controls attention to each word and pixel
according to the semantic regions by manipulating noise images fed to diffusion
models. Experiments show that our method enables more accurate spatial control
than baselines qualitatively and quantitatively.

用不需要额外的训练的方法，通过交叉关注地图和掩膜引导实现文本到图像生成的空间控制。实验结果表明，所提出的方法在定性和定量上比基线方法实现了更准确的空间控制。