We exploit a self-supervised deep multi-task learning framework for
electrocardiogram (ECG) -based emotion recognition. The proposed solution
consists of two stages of learning a) learning ECG representations and b)
learning to classify emotions. ECG representations are learned by a signal
transformation recognition network. The network learns high-level abstract
representations from unlabeled ECG data. Six different signal transformations
are applied to the ECG signals, and transformation recognition is performed as
pretext tasks. Training the model on pretext tasks helps the network learn
spatiotemporal representations that generalize well across different datasets
and different emotion categories. We transfer the weights of the
self-supervised network to an emotion recognition network, where the
convolutional layers are kept frozen and the dense layers are trained with
labelled ECG data. We show that the proposed solution considerably improves the
performance compared to a network trained using fully-supervised learning. New
state-of-the-art results are set in classification of arousal, valence,
affective states, and stress for the four utilized datasets. Extensive
experiments are performed, providing interesting insights into the impact of
using a multi-task self-supervised structure instead of a single-task model, as
well as the optimum level of difficulty required for the pretext
self-supervised tasks.

本研究通过自监督深度多任务学习框架，利用信号变换识别网络学习心电图 (ECG) 表示，进而进行基于 ECG 的情感识别，结果显示该方案在情感分类领域取得显著表现，并针对自监督的多任务结构与预训练任务难度的影响进行了深入研究。