Emotion recognition models using audio input data can enable the development
of interactive systems with applications in mental healthcare, marketing,
gaming, and social media analysis. While the field of affective computing using
audio data is rich, a major barrier to achieve consistently high-performance
models is the paucity of available training labels. Self-supervised learning
(SSL) is a family of methods which can learn despite a scarcity of supervised
labels by predicting properties of the data itself. To understand the utility
of self-supervised learning for audio-based emotion recognition, we have
applied self-supervised learning pre-training to the classification of emotions
from the CMU- MOSEI's acoustic modality. Unlike prior papers that have
experimented with raw acoustic data, our technique has been applied to encoded
acoustic data. Our model is first pretrained to uncover the randomly-masked
timestamps of the acoustic data. The pre-trained model is then fine-tuned using
a small sample of annotated data. The performance of the final model is then
evaluated via several evaluation metrics against a baseline deep learning model
with an identical backbone architecture. We find that self-supervised learning
consistently improves the performance of the model across all metrics. This
work shows the utility of self-supervised learning for affective computing,
demonstrating that self-supervised learning is most useful when the number of
training examples is small, and that the effect is most pronounced for emotions
which are easier to classify such as happy, sad and anger. This work further
demonstrates that self-supervised learning works when applied to embedded
feature representations rather than the traditional approach of pre-training on
the raw input space.

情感识别通过音频输入数据的模型可以实现在心理健康、营销、游戏和社交媒体分析等领域的交互式系统的发展。