Advent of modern deep learning techniques has given rise to advancements in
the field of Speech Emotion Recognition (SER). However, most systems prevalent
in the field fail to generalize to speakers not seen during training. This
study focuses on handling challenges of multilingual SER, specifically on
unseen speakers. We introduce CAMuLeNet, a novel architecture leveraging
co-attention based fusion and multitask learning to address this problem.
Additionally, we benchmark pretrained encoders of Whisper, HuBERT, Wav2Vec2.0,
and WavLM using 10-fold leave-speaker-out cross-validation on five existing
multilingual benchmark datasets: IEMOCAP, RAVDESS, CREMA-D, EmoDB and CaFE and,
release a novel dataset for SER on the Hindi language (BhavVani). CAMuLeNet
shows an average improvement of approximately 8% over all benchmarks on unseen
speakers determined by our cross-validation strategy.

本研究关注于多语言语音情感识别中未见过的说话者，引入了 CAMuLeNet 这一新颖的架构，通过共同注意力融合和多任务学习解决了这一问题，并在多个基准数据集上对预训练的 Whisper、HuBERT、Wav2Vec2.0 和 WavLM 进行基于说话者排除的交叉验证，同时发布了一个针对印地语的新颖数据集 BhavVani。根据交叉验证策略，CAMuLeNet 在未见过的说话者上显示出平均约 8% 的改进。