Conversational emotion recognition (CER) is an important research topic in
human-computer interactions. Although deep learning (DL) based CER approaches
have achieved excellent performance, existing cross-modal feature fusion
methods used in these DL-based approaches either ignore the intra-modal and
inter-modal emotional interaction or have high computational complexity. To
address these issues, this paper develops a novel cross-modal feature fusion
method for the CER task, i.e., the low-rank matching attention method (LMAM).
By setting a matching weight and calculating attention scores between modal
features row by row, LMAM contains fewer parameters than the self-attention
method. We further utilize the low-rank decomposition method on the weight to
make the parameter number of LMAM less than one-third of the self-attention.
Therefore, LMAM can potentially alleviate the over-fitting issue caused by a
large number of parameters. Additionally, by computing and fusing the
similarity of intra-modal and inter-modal features, LMAM can also fully exploit
the intra-modal contextual information within each modality and the
complementary semantic information across modalities (i.e., text, video and
audio) simultaneously. Experimental results on some benchmark datasets show
that LMAM can be embedded into any existing state-of-the-art DL-based CER
methods and help boost their performance in a plug-and-play manner. Also,
experimental results verify the superiority of LMAM compared with other popular
cross-modal fusion methods. Moreover, LMAM is a general cross-modal fusion
method and can thus be applied to other multi-modal recognition tasks, e.g.,
session recommendation and humour detection.

这篇论文提出了一种名为低秩匹配注意力方法的跨模态特征融合方法，可用于会话情感识别任务，并能够充分利用模态间的语义信息和模态内语境信息。实验证实该方法在诸多基准数据集上的表现均优于其他流行的跨模态融合方法。