Traditional approaches in speech emotion recognition, such as LSTM, CNN, RNN,
SVM, and MLP, have limitations such as difficulty capturing long-term
dependencies in sequential data, capturing the temporal dynamics, and
struggling to capture complex patterns and relationships in multimodal data.
This research addresses these shortcomings by proposing an ensemble model that
combines Graph Convolutional Networks (GCN) for processing textual data and the
HuBERT transformer for analyzing audio signals. We found that GCNs excel at
capturing Long-term contextual dependencies and relationships within textual
data by leveraging graph-based representations of text and thus detecting the
contextual meaning and semantic relationships between words. On the other hand,
HuBERT utilizes self-attention mechanisms to capture long-range dependencies,
enabling the modeling of temporal dynamics present in speech and capturing
subtle nuances and variations that contribute to emotion recognition. By
combining GCN and HuBERT, our ensemble model can leverage the strengths of both
approaches. This allows for the simultaneous analysis of multimodal data, and
the fusion of these modalities enables the extraction of complementary
information, enhancing the discriminative power of the emotion recognition
system. The results indicate that the combined model can overcome the
limitations of traditional methods, leading to enhanced accuracy in recognizing
emotions from speech.

本研究提出了一种综合模型，将图卷积网络（GCN）用于处理文本数据并利用 HuBERT 变换器分析音频信号，通过结合 GCN 和 HuBERT 模型，同时分析多模态数据并融合这些模态，提高了情感识别系统的辨别能力。