Image generation from scene description is a cornerstone technique for the
controlled generation, which is beneficial to applications such as content
creation and image editing. In this work, we aim to synthesize images from
scene description with retrieved patches as reference. We propose a
differentiable retrieval module. With the differentiable retrieval module, we
can (1) make the entire pipeline end-to-end trainable, enabling the learning of
better feature embedding for retrieval; (2) encourage the selection of mutually
compatible patches with additional objective functions. We conduct extensive
quantitative and qualitative experiments to demonstrate that the proposed
method can generate realistic and diverse images, where the retrieved patches
are reasonable and mutually compatible.

该论文提出一种利用场景描述和参考补丁合成图像的方法，并使用可微分检索模块实现端到端的培训，以学习更好的特征嵌入用于检索，并通过附加目标函数鼓励选择相互兼容的补丁。经过广泛的定量和定性实验表明，该方法可以生成逼真多样的图像，检索的补丁合理且相互兼容。