Diffusion models have achieved remarkable results in generating high-quality, diverse, and creative images. However, when it comes to text-based image generation, they often fail to capture the intended meaning presented in the text. For instance, a specified object may not be generated, an unnecessary object may be generated, and an adjective may alter objects it was not intended to modify. Moreover, we found that relationships indicating possession between objects are often overlooked. While users' intentions in text are diverse, existing methods tend to specialize in only some aspects of these. In this paper, we propose Predicated Diffusion, a unified framework to express users' intentions. We consider that the root of the above issues lies in the text encoder, which often focuses only on individual words and neglects the logical relationships between them. The proposed method does not solely rely on the text encoder, but instead, represents the intended meaning in the text as propositions using predicate logic and treats the pixels in the attention maps as the fuzzy predicates. This enables us to obtain a differentiable loss function that makes the image fulfill the proposition by minimizing it. When compared to several existing methods, we demonstrated that Predicated Diffusion can generate images that are more faithful to various text prompts, as verified by human evaluators and pretrained image-text models.

在这篇论文中，我们提出了一种称为预测扩散的统一框架来表达用户的意图，通过使用谓词逻辑将文本中的预期含义表示为命题，并将注意力图中的像素视为模糊谓词，以获取使图像满足命题的可微损失函数。与多种现有方法相比，我们证明了预测扩散能够生成更符合不同文本提示的图像，经由人工评估者和预训练图像-文本模型进行验证。

谓词扩散：基于谓词逻辑的文本到图像扩散模型的注意力引导