Dense Retrieval (DR) is now considered as a promising tool to enhance the
memorization capacity of Large Language Models (LLM) such as GPT3 and GPT-4 by
incorporating external memories. However, due to the paradigm discrepancy
between text generation of LLM and DR, it is still an open challenge to
integrate the retrieval and generation tasks in a shared LLM. In this paper, we
propose an efficient LLM-Oriented Retrieval Tuner, namely LMORT, which
decouples DR capacity from base LLM and non-invasively coordinates the
optimally aligned and uniform layers of the LLM towards a unified DR space,
achieving an efficient and effective DR without tuning the LLM itself. The
extensive experiments on six BEIR datasets show that our approach could achieve
competitive zero-shot retrieval performance compared to a range of strong DR
models while maintaining the generation ability of LLM.

提出了一种高效的基于大型语言模型的检索调谐器 (LMORT)，通过将检索和生成任务从共享的大型语言模型中分离出来，将检索容量与基础语言模型分开，并以非侵入性的方式协调语言模型的统一检索空间，实现了高效和有效的检索，同时保持了生成能力。在六个 BEIR 数据集上的广泛实验表明，我们的方法在保持语言模型生成能力的同时，可以达到与一些强大的密集检索模型相媲美的零样本检索性能。