Emotion recognition based on Electroencephalography (EEG) has gained
significant attention and diversified development in fields such as neural
signal processing and affective computing. However, the unique brain anatomy of
individuals leads to non-negligible natural differences in EEG signals across
subjects, posing challenges for cross-subject emotion recognition. While recent
studies have attempted to address these issues, they still face limitations in
practical effectiveness and model framework unity. Current methods often
struggle to capture the complex spatial-temporal dynamics of EEG signals and
fail to effectively integrate multimodal information, resulting in suboptimal
performance and limited generalizability across subjects. To overcome these
limitations, we develop a Pre-trained model based Multimodal Mood Reader for
cross-subject emotion recognition that utilizes masked brain signal modeling
and interlinked spatial-temporal attention mechanism. The model learns
universal latent representations of EEG signals through pre-training on large
scale dataset, and employs Interlinked spatial-temporal attention mechanism to
process Differential Entropy(DE) features extracted from EEG data.
Subsequently, a multi-level fusion layer is proposed to integrate the
discriminative features, maximizing the advantages of features across different
dimensions and modalities. Extensive experiments on public datasets demonstrate
Mood Reader's superior performance in cross-subject emotion recognition tasks,
outperforming state-of-the-art methods. Additionally, the model is dissected
from attention perspective, providing qualitative analysis of emotion-related
brain areas, offering valuable insights for affective research in neural signal
processing.

基于脑电图 (EEG) 的情绪识别引起了重要关注，在神经信号处理和情感计算等领域有了多样化的发展。然而，个体的独特脑解剖结构导致不可忽视的自然差异，在主题间情绪识别中存在挑战。为了克服这些限制，我们开发了一个基于预训练模型的多模式情绪识别器，利用脑信号建模和空间 - 时间注意机制来学习通用的脑电图表示，并在不同维度和模态中最大化功能的优势。广泛的公共数据集上的实验证明了情绪识别器在跨主题情绪识别任务中的卓越性能，优于现有方法。此外，该模型从注意力的角度进行了剖析，为神经信号处理领域的情感研究提供了有价值的洞察。