Knowledge-based visual question answering (VQA) is a vision-language task
that requires an agent to correctly answer image-related questions using
knowledge that is not presented in the given image. It is not only a more
challenging task than regular VQA but also a vital step towards building a
general VQA system. Most existing knowledge-based VQA systems process knowledge
and image information similarly and ignore the fact that the knowledge base
(KB) contains complete information about a triplet, while the extracted image
information might be incomplete as the relations between two objects are
missing or wrongly detected. In this paper, we propose a novel model named
dynamic knowledge memory enhanced multi-step graph reasoning (DMMGR), which
performs explicit and implicit reasoning over a key-value knowledge memory
module and a spatial-aware image graph, respectively. Specifically, the memory
module learns a dynamic knowledge representation and generates a
knowledge-aware question representation at each reasoning step. Then, this
representation is used to guide a graph attention operator over the
spatial-aware image graph. Our model achieves new state-of-the-art accuracy on
the KRVQR and FVQA datasets. We also conduct ablation experiments to prove the
effectiveness of each component of the proposed model.

本文提出一种名为动态知识记忆增强多步图推理（DMMGR）的新型模型，能够在键 - 值知识记忆模块和空间感知图像图上执行显式和隐式推理，并在 KRVQR 和 FVQA 数据集上实现了新的最先进的准确性。