Retrieval-augmented large language models (LLMs) have been remarkably
competent in various NLP tasks. Despite their great success, the knowledge
provided by the retrieval process is not always useful for improving the model
prediction, since in some samples LLMs may already be quite knowledgeable and
thus be able to answer the question correctly without retrieval. Aiming to save
the cost of retrieval, previous work has proposed to determine when to do/skip
the retrieval in a data-aware manner by analyzing the LLMs' pretraining data.
However, these data-aware methods pose privacy risks and memory limitations,
especially when requiring access to sensitive or extensive pretraining data.
Moreover, these methods offer limited adaptability under fine-tuning or
continual learning settings. We hypothesize that token embeddings are able to
capture the model's intrinsic knowledge, which offers a safer and more
straightforward way to judge the need for retrieval without the privacy risks
associated with accessing pre-training data. Moreover, it alleviates the need
to retain all the data utilized during model pre-training, necessitating only
the upkeep of the token embeddings. Extensive experiments and in-depth analyses
demonstrate the superiority of our model-aware approach.

基于检索增强的大型语言模型在各种 NLP 任务中取得了显着的成功。然而，检索过程提供的知识并不总是有助于提高模型的预测能力。为了节省检索成本，先前的工作通过分析语言模型的预训练数据来决定何时执行 / 跳过检索。然而，这些面向数据的方法存在隐私风险和内存限制问题。本文提出基于标记嵌入的模型感知方法，以更安全直接的方式判断是否需要执行检索，同时避免访问预训练数据所带来的隐私风险，并简化了模型预训练过程中数据的保存要求。大量实验证明了我们的模型感知方法的优越性。