Visual Question Answering (VQA) in its ideal form lets us study reasoning in
the joint space of vision and language and serves as a proxy for the AI task of
scene understanding. However, most VQA benchmarks to date are focused on
questions such as simple counting, visual attributes, and object detection that
do not require reasoning or knowledge beyond what is in the image. In this
paper, we address the task of knowledge-based visual question answering and
provide a benchmark, called OK-VQA, where the image content is not sufficient
to answer the questions, encouraging methods that rely on external knowledge
resources. Our new dataset includes more than 14,000 questions that require
external knowledge to answer. We show that the performance of the
state-of-the-art VQA models degrades drastically in this new setting. Our
analysis shows that our knowledge-based VQA task is diverse, difficult, and
large compared to previous knowledge-based VQA datasets. We hope that this
dataset enables researchers to open up new avenues for research in this domain.
See this http URL to download and browse the dataset.

本文提出了名为 OK-VQA 的基于知识的视觉问答基准，要求使用外部知识资源来回答问题，该任务是多样的、困难的和大型的，与以前的基于知识的 VQA 数据集相比具有更高的难度。