Multimodal emotion recognition from physiological signals is receiving an
increasing amount of attention due to the impossibility to control them at will
unlike behavioral reactions, thus providing more reliable information. Existing
deep learning-based methods still rely on extracted handcrafted features, not
taking full advantage of the learning ability of neural networks, and often
adopt a single-modality approach, while human emotions are inherently expressed
in a multimodal way. In this paper, we propose a hypercomplex multimodal
network equipped with a novel fusion module comprising parameterized
hypercomplex multiplications. Indeed, by operating in a hypercomplex domain the
operations follow algebraic rules which allow to model latent relations among
learned feature dimensions for a more effective fusion step. We perform
classification of valence and arousal from electroencephalogram (EEG) and
peripheral physiological signals, employing the publicly available database
MAHNOB-HCI surpassing a multimodal state-of-the-art network. The code of our
work is freely available at this https URL

通过在超复数域中进行操作，该论文提出了一个带有新型融合模块的超复数多模态网络，实现了从生理信号中的分类情感识别，超越了多模态最先进的网络。