End-to-end spoken language understanding (SLU) has recently attracted
increasing interest. Compared to the conventional tandem-based approach that
combines speech recognition and language understanding as separate modules, the
new approach extracts users' intentions directly from the speech signals,
resulting in joint optimization and low latency. Such an approach, however, is
typically designed to process one intention at a time, which leads users to
take multiple rounds to fulfill their requirements while interacting with a
dialogue system. In this paper, we propose a streaming end-to-end framework
that can process multiple intentions in an online and incremental way. The
backbone of our framework is a unidirectional RNN trained with the
connectionist temporal classification (CTC) criterion. By this design, an
intention can be identified when sufficient evidence has been accumulated, and
multiple intentions can be identified sequentially. We evaluate our solution on
the Fluent Speech Commands (FSC) dataset and the intent detection accuracy is
about 97 % on all multi-intent settings. This result is comparable to the
performance of the state-of-the-art non-streaming models, but is achieved in an
online and incremental way. We also employ our model to a keyword spotting task
using the Google Speech Commands dataset and the results are also highly
promising.

本文提出了一种流式端到端框架，采用单向 RNN 和 CTC 标准进行训练，可以在线和增量地处理多个意图，实现了与最先进的非流式模型相当的精度，并在关键词检测任务中表现出高度的前途。

语音理解的流式端到端框架

A Streaming End-to-End Framework For Spoken Language Understanding

In this paper, we summarize recent progresses made in deep learning based
acoustic models and the motivation and insights behind the surveyed techniques.
We first discuss acoustic models that can effectively exploit variable-length
contextual information, such as recurrent neural networks (RNNs), convolutional
neural networks (CNNs), and their various combination with other models. We
then describe acoustic models that are optimized end-to-end with emphasis on
feature representations learned jointly with rest of the system, the
connectionist temporal classification (CTC) criterion, and the attention-based
sequence-to-sequence model. We further illustrate robustness issues in speech
recognition systems, and discuss acoustic model adaptation, speech enhancement
and separation, and robust training strategies. We also cover modeling
techniques that lead to more efficient decoding and discuss possible future
directions in acoustic model research.

本文总结了基于深度学习的声学模型最近取得的进展和技术的动机和见解，讨论了可以有效利用可变长度上下文信息的语音识别模型，如 RNN、CNN 及其与其他模型的组合，以及优化了端对端性能的声学模型和鲁棒性训练策略，同时讨论了语音增强和分离等建模技术。