Existing text-to-image generation approaches have set high standards for
photorealism and text-image correspondence, largely benefiting from web-scale
text-image datasets, which can include up to 5~billion pairs. However,
text-to-image generation models trained on domain-specific datasets, such as
urban scenes, medical images, and faces, still suffer from low text-image
correspondence due to the lack of text-image pairs. Additionally, collecting
billions of text-image pairs for a specific domain can be time-consuming and
costly. Thus, ensuring high text-image correspondence without relying on
web-scale text-image datasets remains a challenging task. In this paper, we
present a novel approach for enhancing text-image correspondence by leveraging
available semantic layouts. Specifically, we propose a Gaussian-categorical
diffusion process that simultaneously generates both images and corresponding
layout pairs. Our experiments reveal that we can guide text-to-image generation
models to be aware of the semantics of different image regions, by training the
model to generate semantic labels for each pixel. We demonstrate that our
approach achieves higher text-image correspondence compared to existing
text-to-image generation approaches in the Multi-Modal CelebA-HQ and the
Cityscapes dataset, where text-image pairs are scarce. Codes are available in
this this https URL

通过利用可用的语义布局，我们提出了一种增强文本 - 图像对应关系的新方法。具体而言，我们提出了一种高斯 - 分类扩散过程，同时生成图像和相应的布局对。我们的实验证明，通过训练模型为每个像素生成语义标签，我们可以引导文本 - 图像生成模型意识到不同图像区域的语义，从而达到比现有方法更高的文本 - 图像对应关系。