Spoken language understanding is typically based on pipeline architectures
including speech recognition and natural language understanding steps. These
components are optimized independently to allow usage of available data, but
the overall system suffers from error propagation. In this paper, we propose a
novel training method that enables pretrained contextual embeddings to process
acoustic features. In particular, we extend it with an encoder of pretrained
speech recognition systems in order to construct end-to-end spoken language
understanding systems. Our proposed method is based on the teacher-student
framework across speech and text modalities that aligns the acoustic and the
semantic latent spaces. Experimental results in three benchmarks show that our
system reaches the performance comparable to the pipeline architecture without
using any training data and outperforms it after fine-tuning with ten examples
per class on two out of three benchmarks.

本文提出了一种新的训练方法，将预先训练的语境嵌入用于处理声学特征，并扩展了预先训练的语音识别系统的编码器，以构建端到端的口语理解系统，实验结果表明，该系统在三个基准测试中达到与流水线结构相当的性能，在没有使用任何训练数据的情况下，在两个基准测试中经过微调每类 10 个示例后优于流水线结构。