In this paper, we propose QACE, a new metric based on Question Answering for Caption Evaluation. QACE generates questions on the evaluated caption and checks its content by asking the questions on either the reference caption or the source image. We first develop QACE-Ref that compares the answers of the evaluated caption to its reference, and report competitive results with the state-of-the-art metrics. To go further, we propose QACE-Img, which asks the questions directly on the image, instead of reference. A Visual-QA system is necessary for QACE-Img. Unfortunately, the standard VQA models are framed as a classification among only a few thousand categories. Instead, we propose Visual-T5, an abstractive VQA system. The resulting metric, QACE-Img is multi-modal, reference-less, and explainable. Our experiments show that QACE-Img compares favorably w.r.t. other reference-less metrics. We will release the pre-trained models to compute QACE.

提出了一种基于问答的新度量方法QACE来评估标题，该度量方法在参考标题或源图像上提出问题并检查其内容。该文介绍了QACE-Ref和QACE-Img两种方法，并提出了基于Visual-T5的多模式、无参考和可解释性的视觉问答系统QACE-Img，相比其他无参考评估度量结果更好。我们将公开预训练模型以计算QACE。

QACE：对图像标题进行评估的提问