Large language models (LLMs) have the remarkable ability to solve new tasks
with just a few examples, but they need access to the right tools. Retrieval
Augmented Generation (RAG) addresses this problem by retrieving a list of
relevant tools for a given task. However, RAG's tool retrieval step requires
all the required information to be explicitly present in the query. This is a
limitation, as semantic search, the widely adopted tool retrieval method, can
fail when the query is incomplete or lacks context. To address this limitation,
we propose Context Tuning for RAG, which employs a smart context retrieval
system to fetch relevant information that improves both tool retrieval and plan
generation. Our lightweight context retrieval model uses numerical,
categorical, and habitual usage signals to retrieve and rank context items. Our
empirical results demonstrate that context tuning significantly enhances
semantic search, achieving a 3.5-fold and 1.5-fold improvement in Recall@K for
context retrieval and tool retrieval tasks respectively, and resulting in an
11.6% increase in LLM-based planner accuracy. Additionally, we show that our
proposed lightweight model using Reciprocal Rank Fusion (RRF) with LambdaMART
outperforms GPT-4 based retrieval. Moreover, we observe context augmentation at
plan generation, even after tool retrieval, reduces hallucination.

利用上下文调整的检索增强生成模型可以通过智能上下文检索系统获取相关信息，从而提高工具检索和计划生成的准确性。其中，语义搜索在不完整或缺乏上下文的情况下容易失效。经验证明，上下文调整显著提高了语义搜索的性能，并且对于上下文检索和工具检索任务分别实现了 3.5 倍和 1.5 倍的提升，同时使基于 LLM 的计划生成的准确性增加了 11.6％。此外，在使用倒数排名融合（RRF）和 LambdaMART 的轻量级模型时，还观察到了计划生成阶段的上下文增强可以减少虚构现象。