The rapid advancements in the development of multimodal large language models
(MLLMs) have consistently led to new breakthroughs on various benchmarks. In
response, numerous challenging and comprehensive benchmarks have been proposed
to more accurately assess the capabilities of MLLMs. However, there is a dearth
of exploration of the higher-order perceptual capabilities of MLLMs. To fill
this gap, we propose the Image Implication understanding Benchmark, II-Bench,
which aims to evaluate the model's higher-order perception of images. Through
extensive experiments on II-Bench across multiple MLLMs, we have made
significant findings. Initially, a substantial gap is observed between the
performance of MLLMs and humans on II-Bench. The pinnacle accuracy of MLLMs
attains 74.8%, whereas human accuracy averages 90%, peaking at an impressive
98%. Subsequently, MLLMs perform worse on abstract and complex images,
suggesting limitations in their ability to understand high-level semantics and
capture image details. Finally, it is observed that most models exhibit
enhanced accuracy when image sentiment polarity hints are incorporated into the
prompts. This observation underscores a notable deficiency in their inherent
understanding of image sentiment. We believe that II-Bench will inspire the
community to develop the next generation of MLLMs, advancing the journey
towards expert artificial general intelligence (AGI). II-Bench is publicly
available at this https URL

通过对多个多模态大型语言模型进行实验，我们发现高级图片感知能力方面存在较大差距，并且这些模型在理解高级语义和捕捉图片细节方面存在限制。此外，当提供图片情感极性提示时，大多数模型表现出更好的准确性，暴露出它们对图片情感的内在理解不足。