Large vision-language models (VLMs) have demonstrated remarkable abilities in
understanding everyday content. However, their performance in the domain of
art, particularly culturally rich art forms, remains less explored. As a pearl
of human wisdom and creativity, art encapsulates complex cultural narratives
and symbolism. In this paper, we offer the Pun Rebus Art Dataset, a multimodal
dataset for art understanding deeply rooted in traditional Chinese culture. We
focus on three primary tasks: identifying salient visual elements, matching
elements with their symbolic meanings, and explanations for the conveyed
messages. Our evaluation reveals that state-of-the-art VLMs struggle with these
tasks, often providing biased and hallucinated explanations and showing limited
improvement through in-context learning. By releasing the Pun Rebus Art
Dataset, we aim to facilitate the development of VLMs that can better
understand and interpret culturally specific content, promoting greater
inclusiveness beyond English-based corpora.

通过发布《谜语艺术数据集》，本文旨在促进大型视觉 - 语言模型（VLM）的发展，以更好地理解和解释具有文化特定内容的艺术，特别是扎根于中国传统文化的艺术。