Emotion Recognition in Conversation~(ERC) across modalities is of vital
importance for a variety of applications, including intelligent healthcare,
artificial intelligence for conversation, and opinion mining over chat history.
The crux of ERC is to model both cross-modality and cross-time interactions
throughout the conversation. Previous methods have made progress in learning
the time series information of conversation while lacking the ability to trace
down the different emotional states of each speaker in a conversation. In this
paper, we propose a recurrent structure called Speaker Information Enhanced
Long-Short Term Memory (SI-LSTM) for the ERC task, where the emotional states
of the distinct speaker can be tracked in a sequential way to enhance the
learning of the emotion in conversation. Further, to improve the learning of
multimodal features in ERC, we utilize a cross-modal attention component to
fuse the features between different modalities and model the interaction of the
important information from different modalities. Experimental results on two
benchmark datasets demonstrate the superiority of the proposed SI-LSTM against
the state-of-the-art baseline methods in the ERC task on multimodal data.

本研究提出了一种名为 “讲话者信息增强的长短时记忆（SI-LSTM）” 的递归结构，用于情感识别，可以在追踪对话中不同讲话者的情感状态的同时，通过交叉模态注意力机制将不同模态之间的信息结合起来，提高多模态数据情感识别任务的学习效果。在两个基准数据集上进行的实验结果表明，所提出的 SI-LSTM 模型在情感识别任务上具有优越的性能表现。