The purpose of emotion recognition in conversation (ERC) is to identify the
emotion category of an utterance based on contextual information. Previous ERC
methods relied on simple connections for cross-modal fusion and ignored the
information differences between modalities, resulting in the model being unable
to focus on modality-specific emotional information. At the same time, the
shared information between modalities was not processed to generate emotions.
Information redundancy problem. To overcome these limitations, we propose a
cross-modal fusion emotion prediction network based on vector connections. The
network mainly includes two stages: the multi-modal feature fusion stage based
on connection vectors and the emotion classification stage based on fused
features. Furthermore, we design a supervised inter-class contrastive learning
module based on emotion labels. Experimental results confirm the effectiveness
of the proposed method, demonstrating excellent performance on the IEMOCAP and
MELD datasets.

基于向量连接的跨模态融合情感预测网络，包括多模态特征融合阶段和基于融合特征的情感分类阶段，同时设计了基于情感标签的监督式跨类对比学习模块，实验证实了该方法的有效性，并在 IEMOCAP 和 MELD 数据集上展现出优异的性能。

通过情感跨模态融合和类间对比学习提升对话中的情感识别

Enhancing Emotion Recognition in Conversation through Emotional  Cross-Modal Fusion and Inter-class Contrastive Learning

Emotion Recognition in Conversation (ERC) has emerged as a research hotspot
in domains such as conversational robots and question-answer systems. How to
efficiently and adequately retrieve contextual emotional cues has been one of
the key challenges in the ERC task. Existing efforts do not fully model the
context and employ complex network structures, resulting in excessive
computational resource overhead without substantial performance improvement. In
this paper, we propose a novel Emotion Recognition Network based on Curriculum
Learning strategy (ERNetCL). The proposed ERNetCL primarily consists of
Temporal Encoder (TE), Spatial Encoder (SE), and Curriculum Learning (CL) loss.
We utilize TE and SE to combine the strengths of previous methods in a
simplistic manner to efficiently capture temporal and spatial contextual
information in the conversation. To simulate the way humans learn curriculum
from easy to hard, we apply the idea of CL to the ERC task to progressively
optimize the network parameters of ERNetCL. At the beginning of training, we
assign lower learning weights to difficult samples. As the epoch increases, the
learning weights for these samples are gradually raised. Extensive experiments
on four datasets exhibit that our proposed method is effective and dramatically
beats other baseline models.

我们提出了一种基于课程学习策略的新型情感识别网络（ERNetCL），它通过结合前期方法的优点以简洁的方式高效地捕捉对话中的时序和空间上下文信息，并利用课程学习的思想逐步优化网络参数。在四个数据集上进行的大量实验表明，我们的方法有效地击败了其他基准模型。