End-to-end (E2E) spoken language understanding (SLU) systems predict
utterance semantics directly from speech using a single model. Previous work in
this area has focused on targeted tasks in fixed domains, where the output
semantic structure is assumed a priori and the input speech is of limited
complexity. In this work we present our approach to developing an E2E model for
generalized SLU in commercial voice assistants (VAs). We propose a fully
differentiable, transformer-based, hierarchical system that can be pretrained
at both the ASR and NLU levels. This is then fine-tuned on both transcription
and semantic classification losses to handle a diverse set of intent and
argument combinations. This leads to an SLU system that achieves significant
improvements over baselines on a complex internal generalized VA dataset with a
43% improvement in accuracy, while still meeting the 99% accuracy benchmark on
the popular Fluent Speech Commands dataset. We further evaluate our model on a
hard test set, exclusively containing slot arguments unseen in training, and
demonstrate a nearly 20% improvement, showing the efficacy of our approach in
truly demanding VA scenarios.

本文介绍了一种基于可预训练的差分可训练模型和使用 Transformer 的层次化系统的语音识别模型，旨在提高其在商用语音助理中的应用性能。实验表明，与基准系统相比，在一些神经网络模型和数据集之上达到了较好的表现。在对数据进行了重新标注并进行了人工评估后，本文提出的方法的语义准确率得到了大幅提高，这显示出本文提出的方法在商用语音助理中的应用价值。