Recent advances in text-to-image diffusion models have achieved remarkable
success in generating high-quality, realistic images from given text prompts.
However, previous methods fail to perform accurate modality alignment between
text concepts and generated images due to the lack of fine-level semantic
guidance that successfully diagnoses the modality discrepancy. In this paper,
we propose FineRewards to improve the alignment between text and images in
text-to-image diffusion models by introducing two new fine-grained semantic
rewards: the caption reward and the Semantic Segment Anything (SAM) reward.
From the global semantic view, the caption reward generates a corresponding
detailed caption that depicts all important contents in the synthetic image via
a BLIP-2 model and then calculates the reward score by measuring the similarity
between the generated caption and the given prompt. From the local semantic
view, the SAM reward segments the generated images into local parts with
category labels, and scores the segmented parts by measuring the likelihood of
each category appearing in the prompted scene via a large language model, i.e.,
Vicuna-7B. Additionally, we adopt an assemble reward-ranked learning strategy
to enable the integration of multiple reward functions to jointly guide the
model training. Adapting results of text-to-image models on the MS-COCO
benchmark show that the proposed semantic reward outperforms other baseline
reward functions with a considerable margin on both visual quality and semantic
similarity with the input prompt. Moreover, by adopting the assemble
reward-ranked learning strategy, we further demonstrate that model performance
is further improved when adapting under the unifying of the proposed semantic
reward with the current image rewards.

本文提出 FineRewards 方法，通过引入两种细粒度语义奖励 ——caption reward 和 SAM reward，从两个语义视角提升 text-to-image 中文本和图像的匹配，实现模态对齐。在使用多个奖励函数指导训练的时候，FineRewards 方法在 MS-COCO 基准测试上优于其他基准奖励函数，并且与当前图像奖励的统一进一步提高了模型性能。