The field of text-conditioned image generation has made unparalleled progress
with the recent advent of latent diffusion models. While remarkable, as the
complexity of given text input increases, the state-of-the-art diffusion models
may still fail in generating images which accurately convey the semantics of
the given prompt. Furthermore, it has been observed that such misalignments are
often left undetected by pretrained multi-modal models such as CLIP. To address
these problems, in this paper we explore a simple yet effective decompositional
approach towards both evaluation and improvement of text-to-image alignment. In
particular, we first introduce a Decompositional-Alignment-Score which given a
complex prompt decomposes it into a set of disjoint assertions. The alignment
of each assertion with generated images is then measured using a VQA model.
Finally, alignment scores for different assertions are combined aposteriori to
give the final text-to-image alignment score. Experimental analysis reveals
that the proposed alignment metric shows significantly higher correlation with
human ratings as opposed to traditional CLIP, BLIP scores. Furthermore, we also
find that the assertion level alignment scores provide a useful feedback which
can then be used in a simple iterative procedure to gradually increase the
expression of different assertions in the final image outputs. Human user
studies indicate that the proposed approach surpasses previous state-of-the-art
by 8.7% in overall text-to-image alignment accuracy. Project page for our paper
is available at this https URL

通过分解式对齐评估和改进文本到图像的对齐效果，并使用 Decompositional-Alignment-Score 和 VQA 模型来测量不同断言的对齐度。实验结果表明，这种对齐度指标与人类评分高度相关，并且断言级别的对齐度评分可用于逐步提高最终图像输出中不同断言的表达。人类用户研究表明，该方法在整体文本到图像对齐准确性方面超过了之前的最先进方法 8.7%。