ExaRanker recently introduced an approach to training information retrieval
(IR) models, incorporating natural language explanations as additional labels.
The method addresses the challenge of limited labeled examples, leading to
improvements in the effectiveness of IR models. However, the initial results
were based on proprietary language models such as GPT-3.5, which posed
constraints on dataset size due to its cost and data privacy. In this paper, we
introduce ExaRanker-Open, where we adapt and explore the use of open-source
language models to generate explanations. The method has been tested using
different LLMs and datasets sizes to better comprehend the effective
contribution of data augmentation. Our findings reveal that incorporating
explanations consistently enhances neural rankers, with benefits escalating as
the LLM size increases. Notably, the data augmentation method proves
advantageous even with large datasets, as evidenced by ExaRanker surpassing the
target baseline by 0.6 nDCG@10 points in our study. To encourage further
advancements by the research community, we have open-sourced both the code and
datasets at this https URL

ExaRanker-Open 使用开源语言模型产生解释，通过数据增强方法不断加强神经排序器，在不同的数据集大小下持续提升信息检索模型的效果，并在研究中以 nDCG@10 点优于目标基准 0.6 点。

ExaRanker-Open：使用开源 LLMs 的 IR 合成解释

ExaRanker-Open: Synthetic Explanation for IR using Open-Source LLMs

Recent work has shown that inducing a large language model (LLM) to generate
explanations prior to outputting an answer is an effective strategy to improve
performance on a wide range of reasoning tasks. In this work, we show that
neural rankers also benefit from explanations. We use LLMs such as GPT-3.5 to
augment retrieval datasets with explanations and train a sequence-to-sequence
ranking model to output a relevance label and an explanation for a given
query-document pair. Our model, dubbed ExaRanker, finetuned on a few thousand
examples with synthetic explanations performs on par with models finetuned on
3x more examples without explanations. Furthermore, the ExaRanker model incurs
no additional computational cost during ranking and allows explanations to be
requested on demand.

本文研究表明：在输出答案之前，引导大型语言模型生成解释是提高推理任务性能的有效策略；本研究还发现神经排名器也受益于解释。我们使用像 GPT-3.5 这样的大型语言模型对检索数据集进行增强，并训练一个序列到序列的排名模型，输出给定查询 - 文档对的相关性标签和解释。我们的模型，ExaRanker，在少量带有合成解释的示例上微调，性能与在 3 倍更多没有解释的示例上微调的模型相当。此外，ExaRanker 模型在排名过程中不产生额外的计算成本，可以按需请求解释。