Visual Dialog requires an agent to engage in a conversation with humans
grounded in an image. Many studies on Visual Dialog focus on the understanding
of the dialog history or the content of an image, while a considerable amount
of commonsense-required questions are ignored. Handling these scenarios depends
on logical reasoning that requires commonsense priors. How to capture relevant
commonsense knowledge complementary to the history and the image remains a key
challenge. In this paper, we propose a novel model by Reasoning with
Multi-structure Commonsense Knowledge (RMK). In our model, the external
knowledge is represented with sentence-level facts and graph-level facts, to
properly suit the scenario of the composite of dialog history and image. On top
of these multi-structure representations, our model can capture relevant
knowledge and incorporate them into the vision and semantic features, via
graph-based interaction and transformer-based fusion. Experimental results and
analysis on VisDial v1.0 and VisDialCK datasets show that our proposed model
effectively outperforms comparative methods.

本文提出了一种基于多结构的常识知识的推理模型，通过句子级事实和图级事实表示外部知识，通过图形交互和变压器融合捕获相关的知识并将其融入视觉和语义特征中，已在 VisDial v1.0 和 VisDialCK 数据集上取得了有效的性能优于比较方法。