Recent advancements in transformer-based speech representation models have
greatly transformed speech processing. However, there has been limited research
conducted on evaluating these models for speech emotion recognition (SER)
across multiple languages and examining their internal representations. This
article addresses these gaps by presenting a comprehensive benchmark for SER
with eight speech representation models and six different languages. We
conducted probing experiments to gain insights into inner workings of these
models for SER. We find that using features from a single optimal layer of a
speech model reduces the error rate by 32\% on average across seven datasets
when compared to systems where features from all layers of speech models are
used. We also achieve state-of-the-art results for German and Persian
languages. Our probing results indicate that the middle layers of speech models
capture the most important emotional information for speech emotion
recognition.

使用八种语音表示模型和六种不同语言，该研究首次全面评估了基于 Transformer 的语音表示模型在跨多种语言的语音情感识别中的性能，并通过探索性实验揭示了这些模型的内部表征。实验结果表明，相较于使用所有层的语音模型特征的系统，仅使用语音模型最优层的特征平均降低了 7 个数据集上的错误率 32％。此外，还在德语和波斯语的语音情感识别方面取得了最新的成果，探测结果表明中间层的语音模型对于捕捉最重要的情感信息至关重要。