Emotion Recognition in Conversations (ERC) has considerable prospects for developing empathetic machines. For multimodal ERC, it is vital to understand context and fuse modality information in conversations. Recent graph-based fusion methods generally aggregate multimodal information by exploring unimodal and cross-modal interactions in a graph. However, they accumulate redundant information at each layer, limiting the context understanding between modalities. In this paper, we propose a novel Multimodal Dynamic Fusion Network (MM-DFN) to recognize emotions by fully understanding multimodal conversational context. Specifically, we design a new graph-based dynamic fusion module to fuse multimodal contextual features in a conversation. The module reduces redundancy and enhances complementarity between modalities by capturing the dynamics of contextual information in different semantic spaces. Extensive experiments on two public benchmark datasets demonstrate the effectiveness and superiority of MM-DFN.

本文提出了一种新的多模态动态融合网络（MM-DFN），用于在对话中全面理解多模态的情境信息，通过设计新的基于图的动态融合模块来减少信息冗余并增强不同语义空间中的上下文信息的补充性，对两个公共基准数据集进行了广泛的实验，证明了MM-DFN的有效性和优越性。

MM-DFN：用于对话情感识别的多模态动态融合网络