Human emotion understanding is pivotal in making conversational technology
mainstream. We view speech emotion understanding as a perception task which is
a more realistic setting. With varying contexts (languages, demographics, etc.)
different share of people perceive the same speech segment as a non-unanimous
emotion. As part of the ACM Multimedia 2023 Computational Paralinguistics
ChallengE (ComParE) in the EMotion Share track, we leverage their rich dataset
of multilingual speakers and multi-label regression target of 'emotion share'
or perception of that emotion. We demonstrate that the training scheme of
different foundation models dictates their effectiveness for tasks beyond
speech recognition, especially for non-semantic speech tasks like emotion
understanding. This is a very complex task due to multilingual speakers,
variability in the target labels, and inherent imbalance in the regression
dataset. Our results show that HuBERT-Large with a self-attention-based
light-weight sequence model provides 4.6% improvement over the reported
baseline.

人类情绪理解对于推动对话技术走向主流至关重要。我们视语音情绪理解为一项感知任务，这是一个更加现实的设置。通过在 ACM Multimedia 2023 计算跨语音信息挑战（ComParE）中利用其丰富的多语种演讲者数据集和 ' 情感共享 ' 多标签回归目标，我们展示了不同基础模型的训练方案对于超越语音识别等非语义语音任务（例如情绪理解）的效果产生重要影响，尤其是针对多语种演讲者、目标标签的可变性以及回归数据集中的固有不平衡性，我们的结果表明，基于自注意力的轻量级序列模型 HuBERT-Large 相较于报告的基准线有 4.6% 的改进。