Retrieval-augmented generation (RAG) has emerged as an approach to augment large language models (LLMs) by reducing their reliance on static knowledge and improving answer factuality. RAG retrieves relevant context snippets and generates an answer based on them. Despite its increasing industrial adoption, systematic exploration of RAG components is lacking, particularly regarding the ideal size of provided context, and the choice of base LLM and retrieval method. To help guide development of robust RAG systems, we evaluate various context sizes, BM25 and semantic search as retrievers, and eight base LLMs. Moving away from the usual RAG evaluation with short answers, we explore the more challenging long-form question answering in two domains, where a good answer has to utilize the entire context. Our findings indicate that final QA performance improves steadily with up to 15 snippets but stagnates or declines beyond that. Finally, we show that different general-purpose LLMs excel in the biomedical domain than the encyclopedic one, and that open-domain evidence retrieval in large corpora is challenging.

本研究针对检索增强生成（RAG）系统在上下文提供大小和基础大语言模型选择方面的缺乏系统性探索进行了研究。我们的新方法评估了不同的上下文大小和检索方法，发现良好的问答表现依赖于最高15个片段，而不同领域的优秀语言模型表现存在差异，尤其是生物医学领域和百科全书领域。研究结果为开发更稳健的RAG系统提供了指导。 

上下文大小和模型选择对检索增强生成系统的影响