Given an image and an associated textual question, the purpose of
Knowledge-Based Visual Question Answering (KB-VQA) is to provide a correct
answer to the question with the aid of external knowledge bases. Prior KB-VQA
models are usually formulated as a retriever-classifier framework, where a
pre-trained retriever extracts textual or visual information from knowledge
graphs and then makes a prediction among the candidates. Despite promising
progress, there are two drawbacks with existing models. Firstly, modeling
question-answering as multi-class classification limits the answer space to a
preset corpus and lacks the ability of flexible reasoning. Secondly, the
classifier merely consider "what is the answer" without "how to get the
answer", which cannot ground the answer to explicit reasoning paths. In this
paper, we confront the challenge of \emph{explainable open-set} KB-VQA, where
the system is required to answer questions with entities at wild and retain an
explainable reasoning path. To resolve the aforementioned issues, we propose a
new retriever-ranker paradigm of KB-VQA, Graph pATH rankER (GATHER for
brevity). Specifically, it contains graph constructing, pruning, and path-level
ranking, which not only retrieves accurate answers but also provides inference
paths that explain the reasoning process. To comprehensively evaluate our
model, we reformulate the benchmark dataset OK-VQA with manually corrected
entity-level annotations and release it as ConceptVQA. Extensive experiments on
real-world questions demonstrate that our framework is not only able to perform
open-set question answering across the whole knowledge base but provide
explicit reasoning path.

知识引导的视觉问题回答（KB-VQA）通过外部知识库的辅助，为图像和相关的文本问题提供正确答案。本文提出了一种名为 Graph pATH rankER（GATHER）的新型 KB-VQA 框架，通过构建图、修剪和路径级别排序，不仅能够准确检索答案，还提供解释推理过程的路径。通过在真实世界问题上的广泛实验，证明了该框架不仅能够在整个知识库上进行开放式问题回答，还能提供明确的推理路径。