The current state-of-the-art large language models (LLMs) are effective in
generating high-quality text and encapsulating a broad spectrum of world
knowledge. However, these models often hallucinate during generation and are
not designed to utilize external information sources. To enable requests to the
external knowledge bases, also called knowledge grounding, retrieval-augmented
LLMs were introduced. For now, their applications have largely involved Open
Domain Question Answering, Abstractive Question Answering, and such. In this
paper, we broaden the scope of retrieval-augmented LLMs by venturing into a new
task - code generation using external entities. For this task, we collect and
publish a new dataset for project-level code generation, where the model should
reuse functions defined in the project during generation. As we show, existing
retrieval-augmented LLMs fail to assign relevance scores between similar entity
names, and to mitigate it, they expand entity names with description context
and append it to the input. In practice, due to the limited context size they
can not accommodate the indefinitely large context of the whole project. To
solve this issue, we propose a novel end-to-end trainable architecture with an
scalable entity retriever injected directly into the LLM decoder. We
demonstrate that our model can outperform common baselines in several
scenarios, including project-level code generation, as well as Bash and SQL
scripting.

该论文通过引入可检索增强的大型语言模型（LLMs），拓展了检索增强 LLMs 的应用领域，以代码生成为例，并提出了一种新颖的可训练架构，将可检索实体注入到 LLM 解码器中，以解决现有模型在相似实体名之间无法分配相关性分数的问题，并在多个场景下超越常见基准模型，包括项目级代码生成、Bash 和 SQL 脚本。

实体增强代码生成

Entity-Augmented Code Generation

Large Language Models (LLMs) have been applied in the speech domain, often
incurring a performance drop due to misaligned between speech and language
representations. To bridge this gap, we propose a joint speech and language
model (SLM) using a Speech2Text adapter, which maps speech into text token
embedding space without speech information loss. Additionally, using a
CTC-based blank-filtering, we can reduce the speech sequence length to that of
text. In speech MultiWoz dataset (DSTC11 challenge), SLM largely improves the
dialog state tracking (DST) performance (24.7% to 28.4% accuracy). Further to
address errors on rare entities, we augment SLM with a Speech2Entity retriever,
which uses speech to retrieve relevant entities, and then adds them to the
original SLM input as a prefix. With this retrieval-augmented SLM (ReSLM), the
DST performance jumps to 34.6% accuracy. Moreover, augmenting the ASR task with
the dialog understanding task improves the ASR performance from 9.4% to 8.5%
WER.

本文介绍了一种使用 Speech2Text 适配器的联合语音和语言模型，用于桥接语音和语言表示之间的差距，并改进对话状态跟踪性能（DST），并通过 Speech2Entity 检索器增强了 SLM 的性能。