Recent diffusion-based generators can produce high-quality images based only
on textual prompts. However, they do not correctly interpret instructions that
specify the spatial layout of the composition. We propose a simple approach
that can achieve robust layout control without requiring training or
fine-tuning the image generator. Our technique, which we call layout guidance,
manipulates the cross-attention layers that the model uses to interface textual
and visual information and steers the reconstruction in the desired direction
given, e.g., a user-specified layout. In order to determine how to best guide
attention, we study the role of different attention maps when generating images
and experiment with two alternative strategies, forward and backward guidance.
We evaluate our method quantitatively and qualitatively with several
experiments, validating its effectiveness. We further demonstrate its
versatility by extending layout guidance to the task of editing the layout and
context of a given real image.

通过提示文本控制生成的图像在空间布局方面存在问题，因此我们提出了一种名为布局引导技术的新方法，该方法可以改进模型的交叉注意层，以实现所需的重建方向，并成功进行了定量和定性的实验验证与扩展。