Most End-to-End SLU methods depend on the pretrained ASR or language model
features for intent prediction. However, other essential information in speech,
such as prosody, is often ignored. Recent research has shown improved results
in classifying dialogue acts by incorporating prosodic information. The margins
of improvement in these methods are minimal as the neural models ignore
prosodic features. In this work, we propose prosody-attention, which uses the
prosodic features differently to generate attention maps across time frames of
the utterance. Then we propose prosody-distillation to explicitly learn the
prosodic information in the acoustic encoder rather than concatenating the
implicit prosodic features. Both the proposed methods improve the baseline
results, and the prosody-distillation method gives an intent classification
accuracy improvement of 8\% and 2\% on SLURP and STOP datasets over the prosody
baseline.

提出了基于韵律注意力的 End-to-End SLU 模型和基于韵律蒸馏的声学编码器模型，通过利用韵律信息进一步改善意图分类任务表现。

运用韵律注意力和蒸馏技术提高端到端 SLU 性能

Improving End-to-End SLU performance with Prosodic Attention and  Distillation

Spoken Language Understanding (SLU) is a core task in most human-machine
interaction systems. With the emergence of smart homes, smart phones and smart
speakers, SLU has become a key technology for the industry. In a classical SLU
approach, an Automatic Speech Recognition (ASR) module transcribes the speech
signal into a textual representation from which a Natural Language
Understanding (NLU) module extracts semantic information. Recently End-to-End
SLU (E2E SLU) based on Deep Neural Networks has gained momentum since it
benefits from the joint optimization of the ASR and the NLU parts, hence
limiting the cascade of error effect of the pipeline architecture. However,
little is known about the actual linguistic properties used by E2E models to
predict concepts and intents from speech input. In this paper, we present a
study identifying the signal features and other linguistic properties used by
an E2E model to perform the SLU task. The study is carried out in the
application domain of a smart home that has to handle non-English (here French)
voice commands. The results show that a good E2E SLU performance does not
always require a perfect ASR capability. Furthermore, the results show the
superior capabilities of the E2E model in handling background noise and
syntactic variation compared to the pipeline model. Finally, a finer-grained
analysis suggests that the E2E model uses the pitch information of the input
signal to identify voice command concepts. The results and methodology outlined
in this paper provide a springboard for further analyses of E2E models in
speech processing.

本文研究了利用深度神经网络的端到端口语理解模型 (E2E SLU) 在非英语智能家居背景下使用的语言特性，证明了优良的 E2E SLU 表现并不总是需要完美的 ASR 能力，并且 E2E 模型相较于传统管道模型在处理背景噪声和语法变异等方面有更出色的性能。该研究还发现，E2E 模型通过音调信息识别语音命令概念。