Most existing image captioning evaluation metrics focus on assigning a single
numerical score to a caption by comparing it with reference captions. However,
these methods do not provide an explanation for the assigned score. Moreover,
reference captions are expensive to acquire. In this paper, we propose FLEUR,
an explainable reference-free metric to introduce explainability into image
captioning evaluation metrics. By leveraging a large multimodal model, FLEUR
can evaluate the caption against the image without the need for reference
captions, and provide the explanation for the assigned score. We introduce
score smoothing to align as closely as possible with human judgment and to be
robust to user-defined grading criteria. FLEUR achieves high correlations with
human judgment across various image captioning evaluation benchmarks and
reaches state-of-the-art results on Flickr8k-CF, COMPOSITE, and Pascal-50S
within the domain of reference-free evaluation metrics. Our source code and
results are publicly available at: this https URL

借助大型多模型，本文提出了一种解释性的无参考度量方法 FLEUR，用以评估图像说明。FLEUR 不需参考说明即能对图像说明进行评估，并提供评分的解释。通过引入评分平滑化以尽可能与人类判断接近，并对用户定义的等级标准具有鲁棒性，FLEUR 在各种图像说明评估基准上与人类判断之间达到了高度相关性，并在 Flickr8k-CF、COMPOSITE 和 Pascal-50S 上取得了最先进的结果，成为了无参考评估度量领域的最新成果。

FLEUR：基于大型多模态模型的图像字幕解释性无参考评估度量

FLEUR: An Explainable Reference-Free Evaluation Metric for Image  Captioning Using a Large Multimodal Model

Model-based, reference-free evaluation metrics have been proposed as a fast
and cost-effective approach to evaluate Natural Language Generation (NLG)
systems. Despite promising recent results, we find evidence that reference-free
evaluation metrics of summarization and dialog generation may be relying on
spurious correlations with measures such as word overlap, perplexity, and
length. We further observe that for text summarization, these metrics have high
error rates when ranking current state-of-the-art abstractive summarization
systems. We demonstrate that these errors can be mitigated by explicitly
designing evaluation metrics to avoid spurious features in reference-free
evaluation.

本研究发现参考无关的自然语言生成系统的评估指标（即用于摘要和对话生成的评估指标）可能会依赖于与诸如单词重叠，困惑度和长度等指标的虚假相关性。同时，我们注意到当评估最先进的抽象化摘要系统时，这些指标具有高错误率，但可以通过明确设计避免参考无关特征进行评估指标以减轻这种错误。