Large language models (LLMs), such as GPT3.5, GPT4 and LLAMA2 perform
surprisingly well and outperform human experts on many tasks. However, in many
domain-specific evaluations, these LLMs often suffer from hallucination
problems due to insufficient training of relevant corpus. Furthermore,
fine-tuning large models may face problems such as the LLMs are not open source
or the construction of high-quality domain instruction is difficult. Therefore,
structured knowledge databases such as knowledge graph can better provide
domain back- ground knowledge for LLMs and make full use of the reasoning and
analysis capabilities of LLMs. In some previous works, LLM was called multiple
times to determine whether the current triplet was suitable for inclusion in
the subgraph when retrieving subgraphs through a question. Especially for the
question that require a multi-hop reasoning path, frequent calls to LLM will
consume a lot of computing power. Moreover, when choosing the reasoning path,
LLM will be called once for each step, and if one of the steps is selected
incorrectly, it will lead to the accumulation of errors in the following steps.
In this paper, we integrated and optimized a pipeline for selecting reasoning
paths from KG based on LLM, which can reduce the dependency on LLM. In
addition, we propose a simple and effective subgraph retrieval method based on
chain of thought (CoT) and page rank which can returns the paths most likely to
contain the answer. We conduct experiments on three datasets: GenMedGPT-5k
[14], WebQuestions [2], and CMCQA [21]. Finally, RoK can demonstrate that using
fewer LLM calls can achieve the same results as previous SOTAs models.

通过在知识图谱（KG）上选择推理路径并结合思维链（CoT）和 PageRank 的子图检索方法，本文整合和优化了一个基于大型语言模型（LLM）的管道，可以减少对 LLM 的依赖。最终，RoK 证明了使用较少的 LLM 调用可以达到相同的结果。