We propose emotion2vec, a universal speech emotion representation model.
emotion2vec is pre-trained on open-source unlabeled emotion data through
self-supervised online distillation, combining utterance-level loss and
frame-level loss during pre-training. emotion2vec outperforms state-of-the-art
pre-trained universal models and emotion specialist models by only training
linear layers for the speech emotion recognition task on the mainstream IEMOCAP
dataset. In addition, emotion2vec shows consistent improvements among 10
different languages of speech emotion recognition datasets. emotion2vec also
shows excellent results on other emotion tasks, such as song emotion
recognition, emotion prediction in conversation, and sentiment analysis.
Comparison experiments, ablation experiments, and visualization comprehensively
demonstrate the universal capability of the proposed emotion2vec. To the best
of our knowledge, emotion2vec is the first universal representation model in
various emotion-related tasks, filling a gap in the field.

我们提出了 emotion2vec，这是一个通用的语音情感表示模型，通过自我监督在线提取方法在开源未标记的情感数据上进行预训练，结合预训练期间的话语级损失和帧级损失。emotion2vec 在主流的 IEMOCAP 数据集上，仅通过训练线性层的方式在语音情感识别任务中胜过最先进的预训练通用模型和情感专家模型。此外，emotion2vec 在 10 种不同语言的语音情感识别数据集中表现出一致的改进。emotion2vec 在其他情感任务上也展示了出色的结果，如歌曲情感识别、对话中的情感预测和情感分析。比较实验、消融实验和可视化全面展示了提出的 emotion2vec 模型的通用性。据我们所知，emotion2vec 是各种与情感相关任务中的第一个通用表示模型，填补了这一领域的空白。