Visual commonsense contains knowledge about object properties, relationships,
and behaviors in visual data. Discovering visual commonsense can provide a more
comprehensive and richer understanding of images, and enhance the reasoning and
decision-making capabilities of computer vision systems. However, the visual
commonsense defined in existing visual commonsense discovery studies is
coarse-grained and incomplete. In this work, we draw inspiration from a
commonsense knowledge base ConceptNet in natural language processing, and
systematically define the types of visual commonsense. Based on this, we
introduce a new task, Visual Commonsense Discovery (VCD), aiming to extract
fine-grained commonsense of different types contained within different objects
in the image. We accordingly construct a dataset (VCDD) from Visual Genome and
ConceptNet for VCD, featuring over 100,000 images and 14 million
object-commonsense pairs. We furthermore propose a generative model (VCDM) that
integrates a vision-language model with instruction tuning to tackle VCD.
Automatic and human evaluations demonstrate VCDM's proficiency in VCD,
particularly outperforming GPT-4V in implicit commonsense discovery. The value
of VCD is further demonstrated by its application to two downstream tasks,
including visual commonsense evaluation and visual question answering. The data
and code will be made available on GitHub.

本研究通过借鉴自然语言处理中常识知识库 ConceptNet 的方法，系统定义了视觉常识的各种类型，并提出了一种新的任务 - 视觉常识发现（VCD），旨在提取图像中不同对象包含的细粒度常识。通过构建包括超过 10 万张图像和 1400 万个对象 - 常识对的数据集（VCDD），并提出了一种将视觉 - 语言模型与指令调整相结合的生成模型（VCDM），其在 VCD 中表现出色，尤其在隐含常识发现方面优于 GPT-4V。VCD 的价值进一步得到了两个下游任务的应用验证，包括视觉常识评估和视觉问答。数据和代码在 GitHub 上可获得。