With the continuous development of deep learning (DL), the task of multimodal
dialogue emotion recognition (MDER) has recently received extensive research
attention, which is also an essential branch of DL. The MDER aims to identify
the emotional information contained in different modalities, e.g., text, video,
and audio, in different dialogue scenes. However, existing research has focused
on modeling contextual semantic information and dialogue relations between
speakers while ignoring the impact of event relations on emotion. To tackle the
above issues, we propose a novel Dialogue and Event Relation-Aware Graph
Convolutional Neural Network for Multimodal Emotion Recognition (DER-GCN)
method. It models dialogue relations between speakers and captures latent event
relations information. Specifically, we construct a weighted multi-relationship
graph to simultaneously capture the dependencies between speakers and event
relations in a dialogue. Moreover, we also introduce a Self-Supervised Masked
Graph Autoencoder (SMGAE) to improve the fusion representation ability of
features and structures. Next, we design a new Multiple Information Transformer
(MIT) to capture the correlation between different relations, which can provide
a better fuse of the multivariate information between relations. Finally, we
propose a loss optimization strategy based on contrastive learning to enhance
the representation learning ability of minority class features. We conduct
extensive experiments on the IEMOCAP and MELD benchmark datasets, which verify
the effectiveness of the DER-GCN model. The results demonstrate that our model
significantly improves both the average accuracy and the f1 value of emotion
recognition.

提出了一种新颖的基于对话和事件关系感知的图卷积神经网络用于多模态情感识别，该方法通过建立加权多关系图来捕捉说话者之间的对话关系和事件关系，设计了多重信息变换器以捕捉不同关系之间的相关性，通过对比学习的损失优化策略增强了少数类特征的表示学习能力。实验证明，DER-GCN 模型在情感识别方面显著提高了平均准确率和 F1 值。

DER-GCN: 多模态对话情感识别中的对话和事件关系感知图卷积神经网络

DER-GCN: Dialogue and Event Relation-Aware Graph Convolutional Neural  Network for Multimodal Dialogue Emotion Recognition

Fully understanding narratives often requires identifying events in the
context of whole documents and modeling the event relations. However,
document-level event extraction is a challenging task as it requires the
extraction of event and entity coreference, and capturing arguments that span
across different sentences. Existing works on event extraction usually confine
on extracting events from single sentences, which fail to capture the
relationships between the event mentions at the scale of a document, as well as
the event arguments that appear in a different sentence than the event trigger.
In this paper, we propose an end-to-end model leveraging Deep Value Networks
(DVN), a structured prediction algorithm, to efficiently capture cross-event
dependencies for document-level event extraction. Experimental results show
that our approach achieves comparable performance to CRF-based models on ACE05,
while enjoys significantly higher computational efficiency.

本文提出了一种采用深层价值网络（Deep Value Networks，DVN）的端到端模型，以有效捕获文档级事件提取中的跨事件依赖关系，并在 ACE05 数据集上获得与基于条件随机场（CRF）模型可比的性能，同时具有更高的计算效率。