Despite the recent progress in speech emotion recognition (SER),
state-of-the-art systems are unable to achieve improved performance in
cross-language settings. In this paper, we propose a Multimodal Dual Attention
Transformer (MDAT) model to improve cross-language SER. Our model utilises
pre-trained models for multimodal feature extraction and is equipped with a
dual attention mechanism including graph attention and co-attention to capture
complex dependencies across different modalities and achieve improved
cross-language SER results using minimal target language data. In addition, our
model also exploits a transformer encoder layer for high-level feature
representation to improve emotion classification accuracy. In this way, MDAT
performs refinement of feature representation at various stages and provides
emotional salient features to the classification layer. This novel approach
also ensures the preservation of modality-specific emotional information while
enhancing cross-modality and cross-language interactions. We assess our model's
performance on four publicly available SER datasets and establish its superior
effectiveness compared to recent approaches and baseline models.

本研究提出了一个利用基于多模态双重注意力变换器模型，结合图注意力与协同注意力，以提高跨语言情感识别性能，在 4 个公开数据集中获得了优秀表现的方法。该方法在高层次特征表示方面利用 Transformer 编码层来提高情感分类精度，并通过在各个阶段对特征表示进行细化以提供情感显著特征，既保留特定模态的情感信息又增强了跨模态和跨语言交互。