In this paper, we perform an exhaustive evaluation of different
representations to address the intent classification problem in a Spoken
Language Understanding (SLU) setup. We benchmark three types of systems to
perform the SLU intent detection task: 1) text-based, 2) lattice-based, and a
novel 3) multimodal approach. Our work provides a comprehensive analysis of
what could be the achievable performance of different state-of-the-art SLU
systems under different circumstances, e.g., automatically- vs.
manually-generated transcripts. We evaluate the systems on the publicly
available SLURP spoken language resource corpus. Our results indicate that
using richer forms of Automatic Speech Recognition (ASR) outputs, namely
word-consensus-networks, allows the SLU system to improve in comparison to the
1-best setup (5.5% relative improvement). However, crossmodal approaches, i.e.,
learning from acoustic and text embeddings, obtains performance similar to the
oracle setup, a relative improvement of 17.8% over the 1-best configuration,
being a recommended alternative to overcome the limitations of working with
automatically generated transcripts.

本文对三种不同的系统进行了测试，分别为基于文本、基于格子以及基于多模态，研究了在不同情况下使用不同系统的表现；实验结果表明，使用更丰富的自动语音识别输出可以提高语言理解系统的性能，而跨模态的方法可以克服使用自动生成的文本时的问题，并获得约 17.8% 的相对改进。