This paper presents a novel Chunking-Free In-Context (CFIC) retrieval
approach, specifically tailored for Retrieval-Augmented Generation (RAG)
systems. Traditional RAG systems often struggle with grounding responses using
precise evidence text due to the challenges of processing lengthy documents and
filtering out irrelevant content. Commonly employed solutions, such as document
chunking and adapting language models to handle longer contexts, have their
limitations. These methods either disrupt the semantic coherence of the text or
fail to effectively address the issues of noise and inaccuracy in evidence
retrieval.
CFIC addresses these challenges by circumventing the conventional chunking
process. It utilizes the encoded hidden states of documents for in-context
retrieval, employing auto-aggressive decoding to accurately identify the
specific evidence text required for user queries, eliminating the need for
chunking. CFIC is further enhanced by incorporating two decoding strategies,
namely Constrained Sentence Prefix Decoding and Skip Decoding. These strategies
not only improve the efficiency of the retrieval process but also ensure that
the fidelity of the generated grounding text evidence is maintained. Our
evaluations of CFIC on a range of open QA datasets demonstrate its superiority
in retrieving relevant and accurate evidence, offering a significant
improvement over traditional methods. By doing away with the need for document
chunking, CFIC presents a more streamlined, effective, and efficient retrieval
solution, making it a valuable advancement in the field of RAG systems.

本文提出了一种新颖的无切块上下文（CFIC）检索方法，专门为检索增强生成（RAG）系统定制。CFIC 通过绕过传统的切块过程，利用文档的编码隐藏状态进行上下文检索，采用自回归解码准确识别用户查询所需的具体证据文本，消除了切块的需求。通过结合约束句前缀解码和跳过解码等两种策略，CFIC 进一步提高了检索过程的效率，并保证了生成的基于证据文本的真实性。对多个开放型 QA 数据集的评估表明，CFIC 在检索相关准确证据方面优于传统方法，为 RAG 系统领域带来了重大改进。CFIC 无需切块，提供了更简化、更有效和更高效的检索解决方案，成为 RAG 系统领域的一个有价值的进步。