Despite their impressive capabilities, large language models (LLMs) often
face challenges such as temporal misalignment and generating hallucinatory
content. Enhancing LLMs with retrieval mechanisms to fetch relevant information
from external sources offers a promising solution. Inspired by the proverb
"Think twice before you act," we propose a dual-angle evaluated
retrieval-augmented generation framework \textit{Think-then-Act}. Unlike
previous approaches that indiscriminately rewrite queries or perform retrieval
regardless of necessity, or generate temporary responses before deciding on
additional retrieval, which increases model generation costs, our framework
employs a two-phase process: (i) assessing the input query for clarity and
completeness to determine if rewriting is necessary; and (ii) evaluating the
model's capability to answer the query and deciding if additional retrieval is
needed. Experimental results on five datasets show that the
\textit{Think-then-Act} framework significantly improves performance. Our
framework demonstrates notable improvements in accuracy and efficiency compared
to existing baselines and performs well in both English and non-English
contexts. Ablation studies validate the optimal model confidence threshold,
highlighting the resource optimization benefits of our approach.

增强大型语言模型的获取机制，提出了一种双重评估的检索增强生成框架 Think-then-Act，通过评估输入查询的清晰度和完整性，判断是否需要重写查询，并评估模型回答查询的能力和是否需要额外的检索，实验证明该框架大幅提高了性能，优于现有基准，并在英语和非英语环境下表现良好，消融研究验证了最佳模型置信阈值，凸显了我们方法的资源优化效益。

先思考后行动：双角度评估检索增强生成

Think-then-Act: A Dual-Angle Evaluated Retrieval-Augmented Generation

Many NLP models gain performance by having access to a knowledge base. A lot
of research has been devoted to devising and improving the way the knowledge
base is accessed and incorporated into the model, resulting in a number of
mechanisms and pipelines. Despite the diversity of proposed mechanisms, there
are patterns in the designs of such systems. In this paper, we systematically
describe the typology of artefacts (items retrieved from a knowledge base),
retrieval mechanisms and the way these artefacts are fused into the model. This
further allows us to uncover combinations of design decisions that had not yet
been tried. Most of the focus is given to language models, though we also show
how question answering, fact-checking and knowledgable dialogue models fit into
this system as well. Having an abstract model which can describe the
architecture of specific models also helps with transferring these
architectures between multiple NLP tasks.

本文系统性地描述了 NLP 模型中知识库及其检索机制的类型、设计模式和用于模型的融合方法，并揭示了尚未尝试的设计决策组合。此外，还展示了问题回答、事实检查和具有知识对话模型在系统中的适用性，并呈现了能够描述不同任务的模型结构的抽象模型。