Comprehending text-rich visual content is paramount for the practical
application of Multimodal Large Language Models (MLLMs), since text-rich
scenarios are ubiquitous in the real world, which are characterized by the
presence of extensive texts embedded within images. Recently, the advent of
MLLMs with impressive versatility has raised the bar for what we can expect
from MLLMs. However, their proficiency in text-rich scenarios has yet to be
comprehensively and objectively assessed, since current MLLM benchmarks
primarily focus on evaluating general visual comprehension. In this work, we
introduce SEED-Bench-2-Plus, a benchmark specifically designed for evaluating
\textbf{text-rich visual comprehension} of MLLMs. Our benchmark comprises 2.3K
multiple-choice questions with precise human annotations, spanning three broad
categories: Charts, Maps, and Webs, each of which covers a wide spectrum of
text-rich scenarios in the real world. These categories, due to their inherent
complexity and diversity, effectively simulate real-world text-rich
environments. We further conduct a thorough evaluation involving 34 prominent
MLLMs (including GPT-4V, Gemini-Pro-Vision and Claude-3-Opus) and emphasize the
current limitations of MLLMs in text-rich visual comprehension. We hope that
our work can serve as a valuable addition to existing MLLM benchmarks,
providing insightful observations and inspiring further research in the area of
text-rich visual comprehension with MLLMs. The dataset and evaluation code can
be accessed at this https URL

我们介绍了 SEED-Bench-2-Plus，这是一个专门设计用于评估 MLLMs 的文本丰富视觉理解的基准，通过涵盖现实世界中的三个广泛类别（图表、地图和网络），它们有效地模拟了复杂多样的文本丰富环境，并强调了当前 MLLMs 在文本丰富视觉理解方面的限制。