Current metrics for text-to-image models typically rely on statistical
metrics which inadequately represent the real preference of humans. Although
recent work attempts to learn these preferences via human annotated images,
they reduce the rich tapestry of human preference to a single overall score.
However, the preference results vary when humans evaluate images with different
aspects. Therefore, to learn the multi-dimensional human preferences, we
propose the Multi-dimensional Preference Score (MPS), the first
multi-dimensional preference scoring model for the evaluation of text-to-image
models. The MPS introduces the preference condition module upon CLIP model to
learn these diverse preferences. It is trained based on our Multi-dimensional
Human Preference (MHP) Dataset, which comprises 918,315 human preference
choices across four dimensions (i.e., aesthetics, semantic alignment, detail
quality and overall assessment) on 607,541 images. The images are generated by
a wide range of latest text-to-image models. The MPS outperforms existing
scoring methods across 3 datasets in 4 dimensions, enabling it a promising
metric for evaluating and improving text-to-image generation.

通过引入偏爱条件模块到 CLIP 模型，我们提出了首个用于评估文本到图像模型的多维度偏好评分模型，称为 Multi-dimensional Preference Score（MPS）。MPS 在我们的 Multi-dimensional Human Preference（MHP）数据集上进行训练，在四个维度（审美、语义一致性、细节质量和整体评估）上获得 918,315 次人类偏好选择，涵盖了 607,541 张由各类最新文本到图像模型生成的图像。MPS 在三个数据集的四个维度上优于现有评分方法，为评估和改进文本到图像生成提供了有希望的指标。