Large language model (LLM) has achieved outstanding performance on various
downstream tasks with its powerful natural language understanding and zero-shot
capability, but LLM still suffers from knowledge limitation. Especially in
scenarios that require long logical chains or complex reasoning, the
hallucination and knowledge limitation of LLM limit its performance in question
answering (QA). In this paper, we propose a novel framework KnowledgeNavigator
to address these challenges by efficiently and accurately retrieving external
knowledge from knowledge graph and using it as a key factor to enhance LLM
reasoning. Specifically, KnowledgeNavigator first mines and enhances the
potential constraints of the given question to guide the reasoning. Then it
retrieves and filters external knowledge that supports answering through
iterative reasoning on knowledge graph with the guidance of LLM and the
question. Finally, KnowledgeNavigator constructs the structured knowledge into
effective prompts that are friendly to LLM to help its reasoning. We evaluate
KnowledgeNavigator on multiple public KGQA benchmarks, the experiments show the
framework has great effectiveness and generalization, outperforming previous
knowledge graph enhanced LLM methods and is comparable to the fully supervised
models.

本文提出了一种名为 KnowledgeNavigator 的新型框架，通过从知识图中高效准确地检索外部知识并将其作为增强 LLM 推理的关键因素，以解决 LLM 在需要长逻辑链或复杂推理的场景中所存在的幻觉和知识限制的问题。通过对给定问题的潜在约束进行挖掘和增强，KnowledgeNavigator 指导推理过程；然后，在 LLM 和问题的指导下，通过对知识图上的迭代推理来检索和过滤支持回答的外部知识；最后，KnowledgeNavigator 将结构化知识构建为对 LLM 有利的有效提示，以帮助其推理。实验证明该框架在多个公开的知识图问答基准测试上具有很高的效果和泛化能力，优于之前的知识图增强 LLM 方法，并且与完全监督模型相媲美。