While large language models excel in a variety of natural language processing
(NLP) tasks, to perform well on spoken language understanding (SLU) tasks, they
must either rely on off-the-shelf automatic speech recognition (ASR) systems
for transcription, or be equipped with an in-built speech modality. This work
focuses on the former scenario, where LLM's accuracy on SLU tasks is
constrained by the accuracy of a fixed ASR system on the spoken input.
Specifically, we tackle speech-intent classification task, where a high
word-error-rate can limit the LLM's ability to understand the spoken intent.
Instead of chasing a high accuracy by designing complex or specialized
architectures regardless of deployment costs, we seek to answer how far we can
go without substantially changing the underlying ASR and LLM, which can
potentially be shared by multiple unrelated tasks. To this end, we propose
prompting the LLM with an n-best list of ASR hypotheses instead of only the
error-prone 1-best hypothesis. We explore prompt-engineering to explain the
concept of n-best lists to the LLM; followed by the finetuning of Low-Rank
Adapters on the downstream tasks. Our approach using n-best lists proves to be
effective on a device-directed speech detection task as well as on a keyword
spotting task, where systems using n-best list prompts outperform those using
1-best ASR hypothesis; thus paving the way for an efficient method to exploit
ASR uncertainty via LLMs for speech-based applications.

利用 n-best 列表提示方法，我们改善了基于大型语言模型的口语理解任务，使其能够更好地理解口语意图并应用于基于语音的应用。

利用大型语言模型攫取 ASR 不确定性

Leveraging Large Language Models for Exploiting ASR Uncertainty

This paper presents novel Bayesian optimisation algorithms for minimum error
rate training of statistical machine translation systems. We explore two
classes of algorithms for efficiently exploring the translation space, with the
first based on N-best lists and the second based on a hypergraph representation
that compactly represents an exponential number of translation options. Our
algorithms exhibit faster convergence and are capable of obtaining lower error
rates than the existing translation model specific approaches, all within a
generic Bayesian optimisation framework. Further more, we also introduce a
random embedding algorithm to scale our approach to sparse high dimensional
feature sets.

本文提出了一种使用贝叶斯优化算法进行最小误差率训练的新型统计机器翻译系统，探索了两类算法来高效地探索翻译空间，其中一类基于 N-best 列表，另一类基于超图表示法，同时介绍了一种随机嵌入算法来扩展我们的方法以处理稀疏高维特征集。