In the Massive Open Online Courses (MOOC) learning scenario, the semantic
information of instructional videos has a crucial impact on learners' emotional
state. Learners mainly acquire knowledge by watching instructional videos, and
the semantic information in the videos directly affects learners' emotional
states. However, few studies have paid attention to the potential influence of
the semantic information of instructional videos on learners' emotional states.
To deeply explore the impact of video semantic information on learners'
emotions, this paper innovatively proposes a multimodal emotion recognition
method by fusing video semantic information and physiological signals. We
generate video descriptions through a pre-trained large language model (LLM) to
obtain high-level semantic information about instructional videos. Using the
cross-attention mechanism for modal interaction, the semantic information is
fused with the eye movement and PhotoPlethysmoGraphy (PPG) signals to obtain
the features containing the critical information of the three modes. The
accurate recognition of learners' emotional states is realized through the
emotion classifier. The experimental results show that our method has
significantly improved emotion recognition performance, providing a new
perspective and efficient method for emotion recognition research in MOOC
learning scenarios. The method proposed in this paper not only contributes to a
deeper understanding of the impact of instructional videos on learners'
emotional states but also provides a beneficial reference for future research
on emotion recognition in MOOC learning scenarios.

探索视频语义信息对学习者情绪的影响，本文提出一种融合视频语义信息和生理信号的多模态情绪识别方法，实现了情绪识别性能的显著提升，为 MOOC 学习情景的情绪识别研究提供了新的视角和高效的方法。