We study few-shot reranking for multi-hop QA (MQA) with open-domain
questions. To alleviate the need for a large number of labeled
question-document pairs for retriever training, we propose PromptRank, which
relies on large language models prompting for multi-hop path reranking.
PromptRank first constructs an instruction-based prompt that includes a
candidate document path and then computes the relevance score between a given
question and the path based on the conditional likelihood of the question given
the path prompt according to a language model. PromptRank yields strong
retrieval performance on HotpotQA with only 128 training examples compared to
state-of-the-art methods trained on thousands of examples -- 73.6 recall@10 by
PromptRank vs. 77.8 by PathRetriever and 77.5 by multi-hop dense retrieval.
Code available at this https URL

本文提出了一种基于大型语言模型的 PromptRank 方法，可用于多跳 QA 的少样本重新排序，该方法构建一个基于指令的提示，在给定问题和路径提示的条件概率下计算给定问题和文档路径之间的相关性得分，并且在 HotpotQA 数据集上具有强大的检索性能。