Large language models (LLMs), such as ChatGPT and GPT-4, are versatile and
can solve different tasks due to their emergent ability and generalizability.
However, LLMs sometimes lack domain-specific knowledge to perform tasks, which
would also cause hallucination during inference. In some previous works,
additional modules like graph neural networks (GNNs) are trained on retrieved
knowledge from external knowledge bases, aiming to mitigate the problem of
lacking domain-specific knowledge. However, incorporating additional modules:
1) would need retraining additional modules when encountering novel domains; 2)
would become a bottleneck since LLMs' strong abilities are not fully utilized
for retrieval. In this paper, we propose a paradigm, termed Knowledge Solver
(KSL), to teach LLMs to search for essential knowledge from external knowledge
bases by harnessing their own strong generalizability. Specifically, we design
a simple yet effective prompt to transform retrieval into a multi-hop decision
sequence, which empowers LLMs with searching knowledge ability in zero-shot
manner. Additionally, KSL is able to provide complete retrieval paths and
therefore increase explainability of LLMs' reasoning processes. We conduct
experiments on three datasets: CommonsenseQA, OpenbookQA, and MedQA-USMLE, and
found that our approach improves LLM baseline performance by a relatively large
margin.

本文提出了一种名为知识解决器（KSL）的范例，通过利用大型语言模型（LLMs）的强大概括能力，教给它们从外部知识库中搜索必要知识，并将检索过程转化为多跳决策序列，增强了 LLMs 的搜索能力，提高了推理过程的可解释性。在三个数据集上的实验证实，我们的方法相对基准模型的表现有很大的提升。