Although Retrieval-Augmented Large Language Models (RALMs) demonstrate their
superiority in terms of factuality, they do not consistently outperform the
original retrieval-free Language Models (LMs). Our experiments reveal that this
example-level performance inconsistency exists not only between
retrieval-augmented and retrieval-free LM but also among different retrievers.
To understand this phenomenon, we investigate the degeneration behavior of
RALMs and theoretically decompose it into four categories. Further analysis
based on our decomposition reveals that the innate difference in knowledge
sources and the unpredictable degeneration of the reader model contribute most
to the inconsistency. Drawing from our analysis, we introduce Ensemble of
Retrievers (EoR), a trainable framework that can adaptively retrieve from
different knowledge sources and effectively decrease unpredictable reader
errors. Our experiments on Open Domain Question Answering show that EoR
substantially improves performance over the RALM with a single retriever by
considerably reducing inconsistent behaviors.

综合检索增强的大型语言模型，在研究表明事实性方面较优越，但并不始终优于原始的无检索语言模型。我们的实验揭示了这种例级性能不一致不仅存在于检索增强和无检索语言模型之间，而且在不同的检索器之间也存在。为了理解这一现象，我们对综合检索增强的大型语言模型进行了退化行为研究，并在理论上将其分解为四个类别。根据我们的分解进一步分析表明，知识源的固有差异和读者模型的不可预测的退化对这种不一致性起到了主要贡献。根据我们的分析，我们引入了可训练的综合检索器框架（EoR），该框架可以从不同的知识源自适应地检索，并且可以有效地减少不可预测的读者错误。我们在开放域问题回答方面的实验证明，EoR 显著改善了单个检索器的 RALM 的性能，并大幅减少了不一致的行为。