The advent of large language models (LLMs) has showcased their efficacy
across various domains, yet they often hallucinate, especially in
knowledge-intensive tasks that require external knowledge sources. To improve
factual accuracy of language models, retrieval-augmented generation (RAG) has
emerged as a popular solution. However, traditional retrieval modules often
rely on large-scale document indexes, which can be disconnected from generative
tasks. Through generative retrieval (GR) approach, language models can achieve
superior retrieval performance by directly generating relevant document
identifiers (DocIDs). However, the relationship between GR and downstream
tasks, as well as the potential of LLMs in GR, remains unexplored. In this
paper, we present a unified language model that utilizes external corpus to
handle various knowledge-intensive tasks by seamlessly integrating generative
retrieval, closed-book generation, and RAG. In order to achieve effective
retrieval and generation through a unified continuous decoding process, we
introduce the following mechanisms: (1) a ranking-oriented DocID decoding
strategy, which improves ranking ability by directly learning from a DocID
ranking list; (2) a continuous generation strategy to facilitate effective and
efficient RAG; (3) well-designed auxiliary DocID understanding tasks to enhance
the model's comprehension of DocIDs and their relevance to downstream tasks.
Our approach is evaluated on the widely used KILT benchmark using two variants
of backbone models: an encoder-decoder T5 model and a decoder-only LLM, Llama2.
Experimental results showcase the superior performance of our models in both
retrieval and downstream knowledge-intensive tasks.

该研究论文介绍了一种统一的语言模型，通过将生成检索、闭书生成和 RAG 有机地结合，利用外部语料库处理各种知识密集型任务。该模型通过引入不同机制实现了有效的检索和生成，并在广泛使用的 KILT 基准测试中展现了出色的性能。

面向知识密集任务的统一语言模型：利用外部语料库

Towards a Unified Language Model for Knowledge-Intensive Tasks Utilizing  External Corpus

Retrieval augmentation, which enhances downstream models by a knowledge
retriever and an external corpus instead of by merely increasing the number of
model parameters, has been successfully applied to many natural language
processing (NLP) tasks such as text classification, question answering and so
on. However, existing methods that separately or asynchronously train the
retriever and downstream model mainly due to the non-differentiability between
the two parts, usually lead to degraded performance compared to end-to-end
joint training.

检索增强通过知识检索器和外部语料库提高下游模型的性能，成功应用于许多自然语言处理（NLP）任务，然而，现有的方法由于两个部分之间的不可微性而分别或异步训练检索器和下游模型，通常导致性能下降，与端到端联合训练相比。