The recent advancements in text-to-image generative models have been
remarkable. Yet, the field suffers from a lack of evaluation metrics that
accurately reflect the performance of these models, particularly lacking
fine-grained metrics that can guide the optimization of the models. In this
paper, we propose EvalAlign, a metric characterized by its accuracy, stability,
and fine granularity. Our approach leverages the capabilities of Multimodal
Large Language Models (MLLMs) pre-trained on extensive datasets. We develop
evaluation protocols that focus on two key dimensions: image faithfulness and
text-image alignment. Each protocol comprises a set of detailed, fine-grained
instructions linked to specific scoring options, enabling precise manual
scoring of the generated images. We Supervised Fine-Tune (SFT) the MLLM to
align closely with human evaluative judgments, resulting in a robust evaluation
model. Our comprehensive tests across 24 text-to-image generation models
demonstrate that EvalAlign not only provides superior metric stability but also
aligns more closely with human preferences than existing metrics, confirming
its effectiveness and utility in model assessment.

本文提出了 EvalAlign，这是一种准确性、稳定性和细粒度特性突出的评估指标，通过利用在大规模数据集上进行预训练的多模式大型语言模型（MLLMs）的能力，通过开发集中在图像忠实度和文本 - 图像对齐的两个关键维度的评估协议以及详细的、细粒度的指导指令，进行生成图像的精确手动评分，使 MLLMs 与人类评估判断紧密对齐，得到了一个强大的评估模型。我们对 24 个文本 - 图像生成模型进行了全面的测试，结果表明 EvalAlign 不仅提供了更好的指标稳定性，而且比现有的指标更接近人类的偏好，从而验证了其在模型评估中的有效性和实用性。