Conversational Emotion Recognition (CER) aims to predict the emotion
expressed by an utterance (referred to as an ``event'') during a conversation.
Existing graph-based methods mainly focus on event interactions to comprehend
the conversational context, while overlooking the direct influence of the
speaker's emotional state on the events. In addition, real-time modeling of the
conversation is crucial for real-world applications but is rarely considered.
Toward this end, we propose a novel graph-based approach, namely Event-State
Interactions infused Heterogeneous Graph Neural Network (ESIHGNN), which
incorporates the speaker's emotional state and constructs a heterogeneous
event-state interaction graph to model the conversation. Specifically, a
heterogeneous directed acyclic graph neural network is employed to dynamically
update and enhance the representations of events and emotional states at each
turn, thereby improving conversational coherence and consistency. Furthermore,
to further improve the performance of CER, we enrich the graph's edges with
external knowledge. Experimental results on four publicly available CER
datasets show the superiority of our approach and the effectiveness of the
introduced heterogeneous event-state interaction graph.

我们提出了一种新颖的基于图的方法，即事件 - 状态交互注入异构图神经网络 (ESIHGNN)，它结合了说话者的情绪状态并构建了一个异构的事件 - 状态相互作用图来建模对话，通过动态更新和增强每个轮次中事件和情绪状态的表示来改善对话的连贯性和一致性。此外，为了进一步提高情感识别性能，我们利用外部知识丰富了图的边缘。在四个公开可用的情感识别数据集上的实验结果表明了我们方法的优越性以及引入的异构事件 - 状态相互作用图的有效性。

ESIHGNN：对话情绪识别的事件 - 状态交互融合异构图神经网络

ESIHGNN: Event-State Interactions Infused Heterogeneous Graph Neural  Network for Conversational Emotion Recognition

Conversational emotion recognition (CER) is an important research topic in
human-computer interactions. Although deep learning (DL) based CER approaches
have achieved excellent performance, existing cross-modal feature fusion
methods used in these DL-based approaches either ignore the intra-modal and
inter-modal emotional interaction or have high computational complexity. To
address these issues, this paper develops a novel cross-modal feature fusion
method for the CER task, i.e., the low-rank matching attention method (LMAM).
By setting a matching weight and calculating attention scores between modal
features row by row, LMAM contains fewer parameters than the self-attention
method. We further utilize the low-rank decomposition method on the weight to
make the parameter number of LMAM less than one-third of the self-attention.
Therefore, LMAM can potentially alleviate the over-fitting issue caused by a
large number of parameters. Additionally, by computing and fusing the
similarity of intra-modal and inter-modal features, LMAM can also fully exploit
the intra-modal contextual information within each modality and the
complementary semantic information across modalities (i.e., text, video and
audio) simultaneously. Experimental results on some benchmark datasets show
that LMAM can be embedded into any existing state-of-the-art DL-based CER
methods and help boost their performance in a plug-and-play manner. Also,
experimental results verify the superiority of LMAM compared with other popular
cross-modal fusion methods. Moreover, LMAM is a general cross-modal fusion
method and can thus be applied to other multi-modal recognition tasks, e.g.,
session recommendation and humour detection.

这篇论文提出了一种名为低秩匹配注意力方法的跨模态特征融合方法，可用于会话情感识别任务，并能够充分利用模态间的语义信息和模态内语境信息。实验证实该方法在诸多基准数据集上的表现均优于其他流行的跨模态融合方法。

基于低秩匹配注意力的跨模态特征融合对话情感识别方法

A Low-rank Matching Attention based Cross-modal Feature Fusion Method  for Conversational Emotion Recognition

Conversational emotion recognition (CER) has attracted increasing interests
in the natural language processing (NLP) community. Different from the vanilla
emotion recognition, effective speaker-sensitive utterance representation is
one major challenge for CER. In this paper, we exploit speaker identification
(SI) as an auxiliary task to enhance the utterance representation in
conversations. By this method, we can learn better speaker-aware contextual
representations from the additional SI corpus. Experiments on two benchmark
datasets demonstrate that the proposed architecture is highly effective for
CER, obtaining new state-of-the-art results on two datasets.

本文探讨了会话情感识别 (CER) 的问题，考虑到说话人的信息对于识别情感的影响，提出利用说话人鉴别任务 (SI) 辅助提升对话中话语的表示。实验结果表明，该方法能够有效提高 CER 的性能，达到了两个基准数据集的最优结果。