Representation learning for speech emotion recognition is challenging due to
labeled data sparsity issue and lack of gold standard references. In addition,
there is much variability from input speech signals, human subjective
perception of the signals and emotion label ambiguity. In this paper, we
propose a machine learning framework to obtain speech emotion representations
by limiting the effect of speaker variability in the speech signals.
Specifically, we propose to disentangle the speaker characteristics from
emotion through an adversarial training network in order to better represent
emotion. Our method combines the gradient reversal technique with an entropy
loss function to remove such speaker information. Our approach is evaluated on
both IEMOCAP and CMU-MOSEI datasets. We show that our method improves speech
emotion classification and increases generalization to unseen speakers.

本文提出了一种基于对抗训练网络来消除发言人特征并更好地表示情感的机器学习框架，在 IEMOCAP 和 CMU-MOSEI 数据集上实验表明该框架能够提高语音情感分类的准确性并增加对未知发言人的泛化能力。