End-to-end Spoken Language Understanding (E2E SLU) has attracted increasing
interest due to its advantages of joint optimization and low latency when
compared to traditionally cascaded pipelines. Existing E2E SLU models usually
follow a two-stage configuration where an Automatic Speech Recognition (ASR)
network first predicts a transcript which is then passed to a Natural Language
Understanding (NLU) module through an interface to infer semantic labels, such
as intent and slot tags. This design, however, does not consider the NLU
posterior while making transcript predictions, nor correct the NLU prediction
error immediately by considering the previously predicted word-pieces. In
addition, the NLU model in the two-stage system is not streamable, as it must
wait for the audio segments to complete processing, which ultimately impacts
the latency of the SLU system. In this work, we propose a streamable multi-task
semantic transducer model to address these considerations. Our proposed
architecture predicts ASR and NLU labels auto-regressively and uses a semantic
decoder to ingest both previously predicted word-pieces and slot tags while
aggregating them through a fusion network. Using an industry scale SLU and a
public FSC dataset, we show the proposed model outperforms the two-stage E2E
SLU model for both ASR and NLU metrics.

本文提出了一种可流式传输的多任务语义转换器模型，通过自回归的方式预测 ASR 和 NLU 标签，并使用语义解码器聚合以前预测的字块和槽标签，从而优化了 E2E SLU 模型，且在 ASR 和 NLU 指标上表现优于传统的两阶段 E2E SLU 模型。