Spoken language understanding (SLU) is a task aiming to extract high-level
semantics from spoken utterances. Previous works have investigated the use of
speech self-supervised models and textual pre-trained models, which have shown
reasonable improvements to various SLU tasks. However, because of the
mismatched modalities between speech signals and text tokens, previous methods
usually need complex designs of the frameworks. This work proposes a simple yet
efficient unsupervised paradigm that connects speech and textual pre-trained
models, resulting in an unsupervised speech-to-semantic pre-trained model for
various tasks in SLU. To be specific, we propose to use unsupervised automatic
speech recognition (ASR) as a connector that bridges different modalities used
in speech and textual pre-trained models. Our experiments show that
unsupervised ASR itself can improve the representations from speech
self-supervised models. More importantly, it is shown as an efficient connector
between speech and textual pre-trained models, improving the performances of
five different SLU tasks. Notably, on spoken question answering, we reach the
state-of-the-art result over the challenging NMSQA benchmark.

该研究提出了一种简单高效的无监督语音到语义预训练模型，使用无监督自动语音识别作为连接器，实现了不同任务的语音语言理解，其中包括最近热门的口语问答任务，并在 NMSQA 基准测试中达到了最新的最优结果。