Question Answering (QA) has been a long-standing research topic in AI and NLP
fields, and a wealth of studies have been conducted to attempt to equip QA
systems with human-level reasoning capability. To approximate the complicated
human reasoning process, state-of-the-art QA systems commonly use pre-trained
language models (LMs) to access knowledge encoded in LMs together with
elaborately designed modules based on Graph Neural Networks (GNNs) to perform
reasoning over knowledge graphs (KGs). However, many problems remain open
regarding the reasoning functionality of these GNN-based modules. Can these
GNN-based modules really perform a complex reasoning process? Are they under-
or over-complicated for QA? To open the black box of GNN and investigate these
problems, we dissect state-of-the-art GNN modules for QA and analyze their
reasoning capability. We discover that even a very simple graph neural counter
can outperform all the existing GNN modules on CommonsenseQA and OpenBookQA,
two popular QA benchmark datasets which heavily rely on knowledge-aware
reasoning. Our work reveals that existing knowledge-aware GNN modules may only
carry out some simple reasoning such as counting. It remains a challenging open
problem to build comprehensive reasoning modules for knowledge-powered QA.

本研究分析使用预训练语言模型和图神经网络进行的问题回答，并发现即使是简单的图神经网络也可以在知识驱动的推理任务中胜过现有的复杂神经网络模块，这表明构建完整的知识驱动的问题回答的推理模块仍然是一个具有挑战性的开放问题。