Diffusion models have shown impressive performance in many domains, including
image generation, time series prediction, and reinforcement learning. The
algorithm demonstrates superior performance over the traditional GAN and
transformer based methods. However, the model's capability to follow natural
language instructions (e.g., spatial relationships between objects, generating
complex scenes) is still unsatisfactory. This has been an important research
area to enhance such capability. Prior works adopt reinforcement learning to
adjust the behavior of the diffusion models. However, RL methods not only
require careful reward design and complex hyperparameter tuning, but also fails
to incorporate rich natural language feedback. In this work, we propose
iterative prompt relabeling (IP-RLDF), a novel algorithm that aligns images to
text through iterative image sampling and prompt relabeling. IP-RLDF first
samples a batch of images conditioned on the text, then relabels the text
prompts of unmatched text-image pairs with classifier feedback. We conduct
thorough experiments on three different models, including SDv2, GLIGEN, and
SDXL, testing their capability to generate images following instructions. With
IP-RLDF, we improved up to 15.22% (absolute improvement) on the challenging
spatial relation VISOR benchmark, demonstrating superior performance compared
to previous RL methods.

我们提出了迭代提示重新标记 (IP-RLDF) 算法，通过迭代图像采样和提示重新标记将图像与文本对齐。IP-RLDF 在三种不同模型上进行了彻底实验，包括 SDv2、GLIGEN 和 SDXL，测试了它们根据指令生成图像的能力，并在具有挑战性的空间关系 VISOR 基准测试中实现了高达 15.22% 的改进，相比之前的 RL 方法表现出更强的性能。