Adaptive retrieval-augmented generation (ARAG) aims to dynamically determine
the necessity of retrieval for queries instead of retrieving indiscriminately
to enhance the efficiency and relevance of the sourced information. However,
previous works largely overlook the evaluation of ARAG approaches, leading to
their effectiveness being understudied. This work presents a benchmark,
RetrievalQA, comprising 1,271 short-form questions covering new world and
long-tail knowledge. The knowledge necessary to answer the questions is absent
from LLMs; therefore, external information must be retrieved to answer
correctly. This makes RetrievalQA a suitable testbed to evaluate existing ARAG
methods. We observe that calibration-based methods heavily rely on threshold
tuning, while vanilla prompting is inadequate for guiding LLMs to make reliable
retrieval decisions. Based on our findings, we propose Time-Aware Adaptive
Retrieval (TA-ARE), a simple yet effective method that helps LLMs assess the
necessity of retrieval without calibration or additional training. The dataset
and code will be available at https://github.com/hyintell/RetrievalQA

动态确定查询的检索是否必要的自适应检索增强生成法（ARAG）旨在提高源信息的效率和相关性，但是以往的研究主要忽略了 ARAG 方法的评估，导致其效果未被充分研究。因此，本研究提出了一个基准测试集 RetrievalQA，包含了 1,271 个关于新颖领域和长尾知识的简短问题。为了正确回答这些问题，需要检索到外部信息。我们观察到基于校准的方法在很大程度上依赖于阈值调整，而简单的提示方法无法可靠地引导语言模型进行检索决策。基于我们的发现，我们提出了一种简单而有效的方法 Time-Aware Adaptive Retrieval（TA-ARE），可以帮助语言模型在不进行校准或额外训练的情况下评估检索的必要性。数据集和代码可在 https://github.com/hyintell/RetrievalQA 上获取。