Diffusion models generate images with an unprecedented level of quality, but how can we freely rearrange image layouts? Recent works generate controllable scenes via learning spatially disentangled latent codes, but these methods do not apply to diffusion models due to their fixed forward process. In this work, we propose SceneDiffusion to optimize a layered scene representation during the diffusion sampling process. Our key insight is that spatial disentanglement can be obtained by jointly denoising scene renderings at different spatial layouts. Our generated scenes support a wide range of spatial editing operations, including moving, resizing, cloning, and layer-wise appearance editing operations, including object restyling and replacing. Moreover, a scene can be generated conditioned on a reference image, thus enabling object moving for in-the-wild images. Notably, this approach is training-free, compatible with general text-to-image diffusion models, and responsive in less than a second.

通过对不同空间布局的场景渲染进行联合去噪，SceneDiffusion方法的关键见解在于可以通过优化扩散采样过程中的分层场景表示来实现空间分离，从而生成支持移动、调整大小、克隆以及逐层外观编辑（包括目标重塑和替换）等广泛空间编辑操作的场景。此外，可以根据参考图像生成场景，从而实现对野外图像中的目标进行移动，该方法无需进行训练，与一般的文本-图像扩散模型兼容，并且响应时间少于一秒。

层叠场景扩散移动任何物体