Most End-to-End (E2E) SLU networks leverage the pre-trained ASR networks but
still lack the capability to understand the semantics of utterances, crucial
for the SLU task. To solve this, recently proposed studies use pre-trained NLU
networks. However, it is not trivial to fully utilize both pre-trained
networks; many solutions were proposed, such as Knowledge Distillation,
cross-modal shared embedding, and network integration with Interface. We
propose a simple and robust integration method for the E2E SLU network with
novel Interface, Continuous Token Interface (CTI), the junctional
representation of the ASR and NLU networks when both networks are pre-trained
with the same vocabulary. Because the only difference is the noise level, we
directly feed the ASR network's output to the NLU network. Thus, we can train
our SLU network in an E2E manner without additional modules, such as
Gumbel-Softmax. We evaluate our model using SLURP, a challenging SLU dataset
and achieve state-of-the-art scores on both intent classification and slot
filling tasks. We also verify the NLU network, pre-trained with Masked Language
Model, can utilize a noisy textual representation of CTI. Moreover, we show our
model can be trained with multi-task learning from heterogeneous data even
after integration with CTI.

本文提出了一种简单而健壮的端到端 SLU 模型所需的连续 Token 接口（CTI），从而使 ASR 和 NLU 网络获得网络集成。 在确保仅使用预训练的网络时，我们在 E2E 方式下训练 SLU 网络，并在多任务学习下对异构数据进行培训，并在意图分类和插槽填充任务上实现最先进的性能。