Large Language Models (LLMs) exhibit remarkable capabilities but are prone to
generating inaccurate or hallucinatory responses. This limitation stems from
their reliance on vast pretraining datasets, making them susceptible to errors
in unseen scenarios. To tackle these challenges, Retrieval-Augmented Generation
(RAG) addresses this by incorporating external, relevant documents into the
response generation process, thus leveraging non-parametric knowledge alongside
LLMs' in-context learning abilities. However, existing RAG implementations
primarily focus on initial input for context retrieval, overlooking the nuances
of ambiguous or complex queries that necessitate further clarification or
decomposition for accurate responses. To this end, we propose learning to
Refine Query for Retrieval Augmented Generation (RQ-RAG) in this paper,
endeavoring to enhance the model by equipping it with capabilities for explicit
rewriting, decomposition, and disambiguation. Our experimental results indicate
that our method, when applied to a 7B Llama2 model, surpasses the previous
state-of-the-art (SOTA) by an average of 1.9\% across three single-hop QA
datasets, and also demonstrates enhanced performance in handling complex,
multi-hop QA datasets. Our code is available at
this https URL

学习改进查询以实现检索增强生成，通过加入外部相关文档，使模型具备显式重写、分解和消歧的能力，并在各种问题回答数据集中取得了优于现有方法的表现。

RQ-RAG: 学习改进用于检索增强生成的查询

RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

Retrieval augmented language models have recently become the standard for
knowledge intensive tasks. Rather than relying purely on latent semantics
within the parameters of large neural models, these methods enlist a
semi-parametric memory to encode an index of knowledge for the model to
retrieve over. Most prior work has employed text passages as the unit of
knowledge, which has high coverage at the cost of interpretability,
controllability, and efficiency. The opposite properties arise in other methods
which have instead relied on knowledge base (KB) facts. At the same time, more
recent work has demonstrated the effectiveness of storing and retrieving from
an index of Q-A pairs derived from text \citep{lewis2021paq}. This approach
yields a high coverage knowledge representation that maintains KB-like
properties due to its representations being more atomic units of information.
In this work we push this line of research further by proposing a
question-answer augmented encoder-decoder model and accompanying pretraining
strategy. This yields an end-to-end system that not only outperforms prior QA
retrieval methods on single-hop QA tasks but also enables compositional
reasoning, as demonstrated by strong performance on two multi-hop QA datasets.
Together, these methods improve the ability to interpret and control the model
while narrowing the performance gap with passage retrieval systems.

本文提出了一种以问答对为基础的编码 - 解码模型，通过该模型的预训练策略，能够在单跳及多跳问题上达到强大性能，并进一步提高了对模型的解释和控制能力，缩小了与段落检索系统之间的性能差距。