Pretrained large Vision-Language models have drawn considerable interest in
recent years due to their remarkable performance. Despite considerable efforts
to assess these models from diverse perspectives, the extent of visual cultural
awareness in the state-of-the-art GPT-4V model remains unexplored. To tackle
this gap, we extensively probed GPT-4V using the MaRVL benchmark dataset,
aiming to investigate its capabilities and limitations in visual understanding
with a focus on cultural aspects. Specifically, we introduced three visual
related tasks, i.e. caption classification, pairwise captioning, and culture
tag selection, to systematically delve into fine-grained visual cultural
evaluation. Experimental results indicate that GPT-4V excels at identifying
cultural concepts but still exhibits weaker performance in low-resource
languages, such as Tamil and Swahili. Notably, through human evaluation, GPT-4V
proves to be more culturally relevant in image captioning tasks than the
original MaRVL human annotations, suggesting a promising solution for future
visual cultural benchmark construction.

探索 GPT-4V 在视觉理解方面的能力和限制，着重关注文化方面，通过使用 MaRVL 基准数据集进行详细调查，实验证明 GPT-4V 在识别文化概念方面表现出色，但在低资源语言上仍然存在较弱的性能。