We introduce RAGAs (Retrieval Augmented Generation Assessment), a framework
for reference-free evaluation of Retrieval Augmented Generation (RAG)
pipelines. RAG systems are composed of a retrieval and an LLM based generation
module, and provide LLMs with knowledge from a reference textual database,
which enables them to act as a natural language layer between a user and
textual databases, reducing the risk of hallucinations. Evaluating RAG
architectures is, however, challenging because there are several dimensions to
consider: the ability of the retrieval system to identify relevant and focused
context passages, the ability of the LLM to exploit such passages in a faithful
way, or the quality of the generation itself. With RAGAs, we put forward a
suite of metrics which can be used to evaluate these different dimensions
\textit{without having to rely on ground truth human annotations}. We posit
that such a framework can crucially contribute to faster evaluation cycles of
RAG architectures, which is especially important given the fast adoption of
LLMs.

我们介绍了 RAGAs（检索增强生成评估）框架，用于无参考评估检索增强生成（RAG）流程，并提出一套可以用来评估不同维度的指标，而无需依赖于人工标注的真值，这对于 RAG 体系结构的快速评估是至关重要的，尤其是考虑到 LLMs 的快速采用。