Automatically evaluating vision-language tasks is challenging, especially
when it comes to reflecting human judgments due to limitations in accounting
for fine-grained details. Although GPT-4V has shown promising results in
various multi-modal tasks, leveraging GPT-4V as a generalist evaluator for
these tasks has not yet been systematically explored. We comprehensively
validate GPT-4V's capabilities for evaluation purposes, addressing tasks
ranging from foundational image-to-text and text-to-image synthesis to
high-level image-to-image translations and multi-images to text alignment. We
employ two evaluation methods, single-answer grading and pairwise comparison,
using GPT-4V. Notably, GPT-4V shows promising agreement with humans across
various tasks and evaluation methods, demonstrating immense potential for
multi-modal LLMs as evaluators. Despite limitations like restricted visual
clarity grading and real-world complex reasoning, its ability to provide
human-aligned scores enriched with detailed explanations is promising for
universal automatic evaluator.

GPT-4V 在多模态任务的普遍评估方面展现出了巨大的潜力，尽管存在一些限制，但其与人类的一致性以及提供详细解释的能力为通用自动评估器提供了希望。