Research in Image Generation has recently made significant progress,
particularly boosted by the introduction of Vision-Language models which are
able to produce high-quality visual content based on textual inputs. Despite
ongoing advancements in terms of generation quality and realism, no methodical
frameworks have been defined yet to quantitatively measure the quality of the
generated content and the adherence with the prompted requests: so far, only
human-based evaluations have been adopted for quality satisfaction and for
comparing different generative methods. We introduce a novel automated method
for Visual Concept Evaluation (ViCE), i.e. to assess consistency between a
generated/edited image and the corresponding prompt/instructions, with a
process inspired by the human cognitive behaviour. ViCE combines the strengths
of Large Language Models (LLMs) and Visual Question Answering (VQA) into a
unified pipeline, aiming to replicate the human cognitive process in quality
assessment. This method outlines visual concepts, formulates image-specific
verification questions, utilizes the Q&A system to investigate the image, and
scores the combined outcome. Although this brave new hypothesis of mimicking
humans in the image evaluation process is in its preliminary assessment stage,
results are promising and open the door to a new form of automatic evaluation
which could have significant impact as the image generation or the image target
editing tasks become more and more sophisticated.

通过模拟人类认知过程，我们提出了一种新的自动化的视觉概念评估方法（ViCE），用于评估生成 / 编辑的图像与相应提示 / 说明之间的一致性，并为图像评分。虽然这种模拟人类在图像评估过程中的新假设正处于初步评估阶段，但结果令人鼓舞，并为一种新形式的自动评估打开了大门，这将在图像生成或图像目标编辑任务变得越来越复杂时产生重要影响。