Food is a rich and varied dimension of cultural heritage, crucial to both
individuals and social groups. To bridge the gap in the literature on the
often-overlooked regional diversity in this domain, we introduce FoodieQA, a
manually curated, fine-grained image-text dataset capturing the intricate
features of food cultures across various regions in China. We evaluate
vision-language Models (VLMs) and large language models (LLMs) on newly
collected, unseen food images and corresponding questions. FoodieQA comprises
three multiple-choice question-answering tasks where models need to answer
questions based on multiple images, a single image, and text-only descriptions,
respectively. While LLMs excel at text-based question answering, surpassing
human accuracy, the open-sourced VLMs still fall short by 41\% on multi-image
and 21\% on single-image VQA tasks, although closed-weights models perform
closer to human levels (within 10\%). Our findings highlight that understanding
food and its cultural implications remains a challenging and under-explored
direction.

对中国各地区的食物文化进行分析，创建了 FoodieQA 数据集，包括图像和文字描述，并评估了视觉 - 语言模型和大型语言模型的表现，发现在多图像和单图像问答任务上，开源的视觉 - 语言模型仍然存在较大的差距，而理解食物文化及其文化含义仍然是一个具有挑战性且尚未深入探索的方向。