Emotional states, as indicators of affect, are pivotal to overall health,
making their accurate prediction before onset crucial. Current studies are
primarily centered on immediate short-term affect detection using data from
wearable and mobile devices. These studies typically focus on objective sensory
measures, often neglecting other forms of self-reported information like
diaries and notes. In this paper, we propose a multimodal deep learning model
for affect status forecasting. This model combines a transformer encoder with a
pre-trained language model, facilitating the integrated analysis of objective
metrics and self-reported diaries. To validate our model, we conduct a
longitudinal study, enrolling college students and monitoring them over a year,
to collect an extensive dataset including physiological, environmental, sleep,
metabolic, and physical activity parameters, alongside open-ended textual
diaries provided by the participants. Our results demonstrate that the proposed
model achieves predictive accuracy of 82.50% for positive affect and 82.76% for
negative affect, a full week in advance. The effectiveness of our model is
further elevated by its explainability.

本研究提出一种多模态深度学习模型，结合了转换器编码器和预训练语言模型，利用客观指标和自我报告日记进行情感状态预测。通过进行为期一年的纵向研究，监测大学生参与者的身体、环境、睡眠、新陈代谢和体力活动参数，以及开放式的文本日记，实证结果表明该模型在提前一周预测积极和消极情感方面的准确率分别为 82.50% 和 82.76%。该模型的解释性进一步提高了其有效性。