In traditional RAG framework, the basic retrieval units are normally short.
The common retrievers like DPR normally work with 100-word Wikipedia
paragraphs. Such a design forces the retriever to search over a large corpus to
find the `needle' unit. In contrast, the readers only need to extract answers
from the short retrieved units. Such an imbalanced `heavy' retriever and
`light' reader design can lead to sub-optimal performance. In order to
alleviate the imbalance, we propose a new framework LongRAG, consisting of a
`long retriever' and a `long reader'. LongRAG processes the entire Wikipedia
into 4K-token units, which is 30x longer than before. By increasing the unit
size, we significantly reduce the total units from 22M to 700K. This
significantly lowers the burden of retriever, which leads to a remarkable
retrieval score: answer recall@1=71% on NQ (previously 52%) and answer
recall@2=72% (previously 47%) on HotpotQA (full-wiki). Then we feed the top-k
retrieved units ($\approx$ 30K tokens) to an existing long-context LLM to
perform zero-shot answer extraction. Without requiring any training, LongRAG
achieves an EM of 62.7% on NQ, which is the best known result. LongRAG also
achieves 64.3% on HotpotQA (full-wiki), which is on par of the SoTA model. Our
study offers insights into the future roadmap for combining RAG with
long-context LLMs.

在传统的 RAG 框架中，检索单元通常很短，而我们提出的 LongRAG 框架则将整个维基百科以 4K-token 为单位处理，通过增加单元大小和减少总单元数量，降低了检索器的负担，并在没有任何训练的情况下实现了最佳结果，这为将 RAG 与长文本语言模型相结合的未来发展提供了启示。