Large Language Models (LLMs) often struggle with hallucinations and outdated
information. To address this, Information Retrieval (IR) systems can be
employed to augment LLMs with up-to-date knowledge. However, existing IR
techniques contain deficiencies, posing a performance bottleneck. Given the
extensive array of IR systems, combining diverse approaches presents a viable
strategy. Nevertheless, prior attempts have yielded restricted efficacy. In
this work, we propose an approach that leverages learning-to-rank techniques to
combine heterogeneous IR systems. We demonstrate the method on two Retrieval
Question Answering (ReQA) tasks. Our empirical findings exhibit a significant
performance enhancement, outperforming previous approaches and achieving
state-of-the-art results on ReQA SQuAD.

使用学习排序技术结合不同的信息检索系统，解决大型语言模型中幻觉和过时信息的问题，提高检索问题回答任务的性能，并在 SQuAD 数据集上取得了最先进的结果。

MrRank：通过多结果排名模型提升问答检索系统

MrRank: Improving Question Answering Retrieval System through  Multi-Result Ranking Model

Retrieval-augmented language models show promise in addressing issues like
outdated information and hallucinations in language models (LMs). However,
current research faces two main problems: 1) determining what information to
retrieve, and 2) effectively combining retrieved information during generation.
We argue that valuable retrieved information should not only be related to the
current source text but also consider the future target text, given the nature
of LMs that model future tokens. Moreover, we propose that aggregation using
latent variables derived from a compact latent space is more efficient than
utilizing explicit raw text, which is limited by context length and susceptible
to noise. Therefore, we introduce RegaVAE, a retrieval-augmented language model
built upon the variational auto-encoder (VAE). It encodes the text corpus into
a latent space, capturing current and future information from both source and
target text. Additionally, we leverage the VAE to initialize the latent space
and adopt the probabilistic form of the retrieval generation paradigm by
expanding the Gaussian prior distribution into a Gaussian mixture distribution.
Theoretical analysis provides an optimizable upper bound for RegaVAE.
Experimental results on various datasets demonstrate significant improvements
in text generation quality and hallucination removal.

检索增强语言模型通过引入隐变量聚合检索信息解决了过时信息和虚构问题，并以 RegaVAE 作为一个例子，展示了在文本生成质量和幻觉去除方面的显著改进。