The EMPATHIC project aimed to design an emotionally expressive virtual coach
capable of engaging healthy seniors to improve well-being and promote
independent aging. One of the core aspects of the system is its human sensing
capabilities, allowing for the perception of emotional states to provide a
personalized experience. This paper outlines the development of the emotion
expression recognition module of the virtual coach, encompassing data
collection, annotation design, and a first methodological approach, all
tailored to the project requirements. With the latter, we investigate the role
of various modalities, individually and combined, for discrete emotion
expression recognition in this context: speech from audio, and facial
expressions, gaze, and head dynamics from video. The collected corpus includes
users from Spain, France, and Norway, and was annotated separately for the
audio and video channels with distinct emotional labels, allowing for a
performance comparison across cultures and label types. Results confirm the
informative power of the modalities studied for the emotional categories
considered, with multimodal methods generally outperforming others (around 68%
accuracy with audio labels and 72-74% with video labels). The findings are
expected to contribute to the limited literature on emotion recognition applied
to older adults in conversational human-machine interaction.

EMPATHIC 项目的研究旨在设计一款情感表达丰富的虚拟教练，能够吸引健康老年人提高幸福感并促进独立老龄化。本文概述了虚拟教练的情感表达识别模块的开发，包括数据收集、标注设计和针对项目需求的第一种方法论。结果表明，研究的多模态方法在对老年人进行情感分类时具有信息量，并且通常优于其他方法（音频标签准确率约为 68％，视频标签准确率为 72-74％）。这些发现有助于有限的关于情感识别在老年人与人工智能交互中的文献。