In the Retrieval-Augmented Generation (RAG) system, advanced Large Language
Models (LLMs) have emerged as effective Query Likelihood Models (QLMs) in an
unsupervised way, which re-rank documents based on the probability of
generating the query given the content of a document. However, directly
prompting LLMs to approximate QLMs inherently is biased, where the estimated
distribution might diverge from the actual document-specific distribution. In
this study, we introduce a novel framework, $\mathrm{UR^3}$, which leverages
Bayesian decision theory to both quantify and mitigate this estimation bias.
Specifically, $\mathrm{UR^3}$ reformulates the problem as maximizing the
probability of document generation, thereby harmonizing the optimization of
query and document generation probabilities under a unified risk minimization
objective. Our empirical results indicate that $\mathrm{UR^3}$ significantly
enhances re-ranking, particularly in improving the Top-1 accuracy. It benefits
the QA tasks by achieving higher accuracy with fewer input documents.

利用贝叶斯决策理论提出的 UR^3 框架有效地改进了 Retrieval-Augmented Generation 系统的 Top-1 准确率，通过最大化文档生成的概率在统一的风险最小化目标下优化查询和文档生成概率。

优化零样本 LLM 重排序器的风险最小化

Improving Zero-shot LLM Re-Ranker with Risk Minimization

In the field of information retrieval, Query Likelihood Models (QLMs) rank
documents based on the probability of generating the query given the content of
a document. Recently, advanced large language models (LLMs) have emerged as
effective QLMs, showcasing promising ranking capabilities. This paper focuses
on investigating the genuine zero-shot ranking effectiveness of recent LLMs,
which are solely pre-trained on unstructured text data without supervised
instruction fine-tuning. Our findings reveal the robust zero-shot ranking
ability of such LLMs, highlighting that additional instruction fine-tuning may
hinder effectiveness unless a question generation task is present in the
fine-tuning dataset. Furthermore, we introduce a novel state-of-the-art ranking
system that integrates LLM-based QLMs with a hybrid zero-shot retriever,
demonstrating exceptional effectiveness in both zero-shot and few-shot
scenarios. We make our codebase publicly available at
this https URL

本研究针对最近的大型语言模型探究了原本的零样本排名效果，发现这些仅基于非结构化文本数据进行预训练且没有经过监督指导微调的模型具备强大的零样本排名能力。此外，我们还提出了一个新颖的最先进排名系统，将基于大型语言模型的查询似然模型与混合式零样本检索器相结合，无论是在零样本还是少样本场景下都表现出卓越的效果。我们在 this https URL 上公开了我们的代码库。