Providing external knowledge to Large Language Models (LLMs) is a key point
for using these models in real-world applications for several reasons, such as
incorporating up-to-date content in a real-time manner, providing access to
domain-specific knowledge, and contributing to hallucination prevention. The
vector database-based Retrieval Augmented Generation (RAG) approach has been
widely adopted to this end. Thus, any part of external knowledge can be
retrieved and provided to some LLM as the input context. Despite RAG approach's
success, it still might be unfeasible for some applications, because the
context retrieved can demand a longer context window than the size supported by
LLM. Even when the context retrieved fits into the context window size, the
number of tokens might be expressive and, consequently, impact costs and
processing time, becoming impractical for most applications. To address these,
we propose CRAG, a novel approach able to effectively reduce the number of
prompting tokens without degrading the quality of the response generated
compared to a solution using RAG. Through our experiments, we show that CRAG
can reduce the number of tokens by at least 46\%, achieving more than 90\% in
some cases, compared to RAG. Moreover, the number of tokens with CRAG does not
increase considerably when the number of reviews analyzed is higher, unlike
RAG, where the number of tokens is almost 9x higher when there are 75 reviews
compared to 4 reviews.

提供外部知识给大规模语言模型是实际应用中的关键点之一，为了融入实时的内容、提供特定领域知识以及预防生成虚构内容。通过数据库向量检索增强生成（RAG）方法已经被广泛采用，然而在某些应用中，使用 RAG 方法可能不可行，因为所检索的上下文可能需要比 LLM 支持的上下文窗口更长。为了解决这个问题，我们提出了一种名为 CRAG 的新方法，能够有效减少提示词数量，而不会降低生成的响应质量与使用 RAG 的解决方案相比。通过实验证明，与 RAG 相比，CRAG 方法可以至少减少 46% 的词数量，在某些情况下甚至能达到 90% 以上。此外，与 RAG 相比，CRAG 的词数在分析评论数增加时不会显著增加，当评论数从 4 个增加到 75 个时，RAG 的词数几乎增加了 9 倍。