Emotion recognition is a challenging and actively-studied research area that
plays a critical role in emotion-aware human-computer interaction systems. In a
multimodal setting, temporal alignment between different modalities has not
been well investigated yet. This paper presents a new model named as Gated
Bidirectional Alignment Network (GBAN), which consists of an attention-based
bidirectional alignment network over LSTM hidden states to explicitly capture
the alignment relationship between speech and text, and a novel group gated
fusion (GGF) layer to integrate the representations of different modalities. We
empirically show that the attention-aligned representations outperform the
last-hidden-states of LSTM significantly, and the proposed GBAN model
outperforms existing state-of-the-art multimodal approaches on the IEMOCAP
dataset.

本文提出了一种新的模型 —— 门控双向对齐网络（GBAN），该模型利用注意力机制的双向对齐网络和一种新颖的分组门控融合层来明确捕捉语音和文本之间的对齐关系，并在不同模态中集成表示，从而在 IEMOCAP 数据集上优于现有的最先进的多模态方法。