The task of reasoning over Knowledge Graphs (KGs) poses a significant
challenge for Large Language Models (LLMs) due to the complex structure and
large amounts of irrelevant information. Existing LLM reasoning methods
overlook the importance of compositional learning on KG to supply with precise
knowledge. Besides, the fine-tuning and frequent interaction with LLMs incur
substantial time and resource costs. This paper focuses on the Question
Answering over Knowledge Graph (KGQA) task and proposes an
Explore-then-Determine (EtD) framework that synergizes LLMs with graph neural
networks (GNNs) for reasoning over KGs. The Explore stage employs a lightweight
GNN to explore promising candidates and relevant fine-grained knowledge to the
questions, while the Determine stage utilizes the explored information to
construct a knowledge-enhanced multiple-choice prompt, guiding a frozen LLM to
determine the final answer. Extensive experiments on three benchmark KGQA
datasets demonstrate that EtD achieves state-of-the-art performance and
generates faithful reasoning results.

此论文提出了一个 Explore-then-Determine（EtD）框架，将 Large Language Models（LLMs）与图神经网络（GNNs）相结合，用于在知识图谱上进行推理。在 Explore 阶段，使用轻量级 GNN 探索有希望的候选项和与问题相关的细粒度知识，而在 Determine 阶段，利用探索的信息构建一个知识增强的多项选择提示，指导冻结的 LLM 确定最终答案。对三个基准 KGQA 数据集进行的大量实验表明，EtD 实现了最先进的性能并生成了忠实的推理结果。

探索然后确定：用于知识图谱推理的 GNN-LLM 协同框架

Explore then Determine: A GNN-LLM Synergy Framework for Reasoning over  Knowledge Graph

Question Answering over Knowledge Graph (KGQA) aims to seek answer entities
for the natural language question from a large-scale Knowledge Graph~(KG). To
better perform reasoning on KG, recent work typically adopts a pre-trained
language model~(PLM) to model the question, and a graph neural network~(GNN)
based module to perform multi-hop reasoning on the KG. Despite the
effectiveness, due to the divergence in model architecture, the PLM and GNN are
not closely integrated, limiting the knowledge sharing and fine-grained feature
interactions. To solve it, we aim to simplify the above two-module approach,
and develop a more capable PLM that can directly support subgraph reasoning for
KGQA, namely ReasoningLM. In our approach, we propose a subgraph-aware
self-attention mechanism to imitate the GNN for performing structured
reasoning, and also adopt an adaptation tuning strategy to adapt the model
parameters with 20,000 subgraphs with synthesized questions. After adaptation,
the PLM can be parameter-efficient fine-tuned on downstream tasks. Experiments
show that ReasoningLM surpasses state-of-the-art models by a large margin, even
with fewer updated parameters and less training data. Our codes and data are
publicly available at~https://github.com/RUCAIBox/ReasoningLM.

Question Answering over Knowledge Graph (KGQA) aims to seek answer entities for the natural language question from a large-scale Knowledge Graph. To better perform reasoning on KG, recent work typically adopts a pre-trained language model (PLM) and a graph neural network (GNN) module, but these are not closely integrated. This paper proposes ReasoningLM, a more capable PLM that directly supports subgraph reasoning for KGQA, outperforming state-of-the-art models.