Efficiently capturing consistent and complementary semantic features in a
multimodal conversation context is crucial for Multimodal Emotion Recognition
in Conversation (MERC). Existing methods mainly use graph structures to model
dialogue context semantic dependencies and employ Graph Neural Networks (GNN)
to capture multimodal semantic features for emotion recognition. However, these
methods are limited by some inherent characteristics of GNN, such as
over-smoothing and low-pass filtering, resulting in the inability to learn
long-distance consistency information and complementary information
efficiently. Since consistency and complementarity information correspond to
low-frequency and high-frequency information, respectively, this paper revisits
the problem of multimodal emotion recognition in conversation from the
perspective of the graph spectrum. Specifically, we propose a
Graph-Spectrum-based Multimodal Consistency and Complementary collaborative
learning framework GS-MCC. First, GS-MCC uses a sliding window to construct a
multimodal interaction graph to model conversational relationships and uses
efficient Fourier graph operators to extract long-distance high-frequency and
low-frequency information, respectively. Then, GS-MCC uses contrastive learning
to construct self-supervised signals that reflect complementarity and
consistent semantic collaboration with high and low-frequency signals, thereby
improving the ability of high and low-frequency information to reflect real
emotions. Finally, GS-MCC inputs the collaborative high and low-frequency
information into the MLP network and softmax function for emotion prediction.
Extensive experiments have proven the superiority of the GS-MCC architecture
proposed in this paper on two benchmark data sets.

通过图谱视角，本文提出了基于图谱的多模态一致性和互补协同学习框架 GS-MCC，通过使用滑动窗口构建多模态交互图、利用高频和低频信息提取方法来反映长程一致性和互补性信息，并通过对比学习构建自监督信号，以提高高频和低频信息对实际情感的反映能力，最后将其输入 MLP 网络和 softmax 函数进行情感预测。在两个基准数据集上的广泛实验证明了 GS-MCC 架构的优越性。