One of the key issues of Visual Question Answering (VQA) is to reason with
semantic clues in the visual content under the guidance of the question, how to
model relational semantics still remains as a great challenge. To fully capture
visual semantics, we propose to reason over a structured visual representation
- scene graph, with embedded objects and inter-object relationships. This shows
great benefit over vanilla vector representations and implicit visual
relationship learning. Based on existing visual relationship models, we propose
a visual relationship encoder that projects visual relationships into a learned
deep semantic space constrained by visual context and language priors. Upon the
constructed graph, we propose a Scene Graph Convolutional Network (SceneGCN) to
jointly reason the object properties and relational semantics for the correct
answer. We demonstrate the model's effectiveness and interpretability on the
challenging GQA dataset and the classical VQA 2.0 dataset, remarkably achieving
state-of-the-art 54.56% accuracy on GQA compared to the existing best model.

使用结构化可视化表征 —— 场景图，通过可视上下文和语言因素进行关系编码，使用场景图卷积网络实现对对象属性和关系语义的联合推理来回答视觉问题，从而在 GQA 数据集上实现了 54.56％的最高准确率。