This paper introduces innovative solutions to enhance spatial controllability
in diffusion models reliant on text queries. We present two key innovations:
Vision Guidance and the Layered Rendering Diffusion (LRDiff) framework. Vision
Guidance, a spatial layout condition, acts as a clue in the perturbed
distribution, greatly narrowing down the search space, to focus on the image
sampling process adhering to the spatial layout condition. The LRDiff framework
constructs an image-rendering process with multiple layers, each of which
applies the vision guidance to instructively estimate the denoising direction
for a single object. Such a layered rendering strategy effectively prevents
issues like unintended conceptual blending or mismatches, while allowing for
more coherent and contextually accurate image synthesis. The proposed method
provides a more efficient and accurate means of synthesising images that align
with specific spatial and contextual requirements. We demonstrate through our
experiments that our method provides better results than existing techniques
both quantitatively and qualitatively. We apply our method to three practical
applications: bounding box-to-image, semantic mask-to-image and image editing.

本文介绍了增强依赖于文本查询的扩散模型的空间可控性的创新解决方案，提出了两个关键创新：视觉引导和分层渲染扩散（LRDiff）框架。该方法通过视觉引导和分层渲染策略实现更高效和准确的图像综合，用于满足特定的空间和上下文需求，并通过实验证明其在定量和定性上优于现有技术。