Speech emotion recognition (SER) is essential for enhancing human-computer
interaction in speech-based applications. Despite improvements in specific
emotional datasets, there is still a research gap in SER's capability to
generalize across real-world situations. In this paper, we investigate
approaches to generalize the SER system across different emotion datasets. In
particular, we incorporate 11 emotional speech datasets and illustrate a
comprehensive benchmark on the SER task. We also address the challenge of
imbalanced data distribution using over-sampling methods when combining SER
datasets for training. Furthermore, we explore various evaluation protocols for
adeptness in the generalization of SER. Building on this, we explore the
potential of Whisper for SER, emphasizing the importance of thorough
evaluation. Our approach is designed to advance SER technology by integrating
speaker-independent methods.

通过整合多个数据集、解决数据分布不均衡以及评估协议，本研究旨在推进语音情感识别技术的发展，以实现人机交互的增强。

如何在数据集间推广 SER 模型？一个全面的基准测试

What Does it Take to Generalize SER Model Across Datasets? A  Comprehensive Benchmark

Real-world datasets often exhibit imbalanced data distribution, where certain
class levels are severely underrepresented. In such cases, traditional pattern
classifiers have shown a bias towards the majority class, impeding accurate
predictions for the minority class. This paper introduces an imbalanced
data-oriented approach using probabilistic neural networks (PNNs) with a skew
normal probability kernel to address this major challenge. PNNs are known for
providing probabilistic outputs, enabling quantification of prediction
confidence and uncertainty handling. By leveraging the skew normal
distribution, which offers increased flexibility, particularly for imbalanced
and non-symmetric data, our proposed Skew Probabilistic Neural Networks
(SkewPNNs) can better represent underlying class densities. To optimize the
performance of the proposed approach on imbalanced datasets, hyperparameter
fine-tuning is imperative. To this end, we employ a population-based heuristic
algorithm, Bat optimization algorithms, for effectively exploring the
hyperparameter space. We also prove the statistical consistency of the density
estimates which suggests that the true distribution will be approached smoothly
as the sample size increases. Experimental simulations have been conducted on
different synthetic datasets, comparing various benchmark-imbalanced learners.
Our real-data analysis shows that SkewPNNs substantially outperform
state-of-the-art machine learning methods for both balanced and imbalanced
datasets in most experimental settings.

利用偏态概率神经网络（SkewPNNs）和超参数微调来解决实际数据集中的类别不平衡问题，并证明其在平衡和不平衡数据集上优于现有的机器学习方法。

不平衡数据学习的倾斜概率神经网络

Skew Probabilistic Neural Networks for Learning from Imbalanced Data

A common practice in metric learning is to train and test an embedding model
for each dataset. This dataset-specific approach fails to simulate real-world
scenarios that involve multiple heterogeneous distributions of data. In this
regard, we introduce a novel metric learning paradigm, called Universal Metric
Learning (UML), which learns a unified distance metric capable of capturing
relations across multiple data distributions. UML presents new challenges, such
as imbalanced data distribution and bias towards dominant distributions. To
address these challenges, we propose Parameter-efficient Universal Metric
leArning (PUMA), which consists of a pre-trained frozen model and two
additional modules, stochastic adapter and prompt pool. These modules enable to
capture dataset-specific knowledge while avoiding bias towards dominant
distributions. Additionally, we compile a new universal metric learning
benchmark with a total of 8 different datasets. PUMA outperformed the
state-of-the-art dataset-specific models while using about 69 times fewer
trainable parameters.

我们引入了一种名为通用度量学习（UML）的新型度量学习范式，它能够学习到能够捕捉多个数据分布关系的统一距离度量。通过使用预训练模型和两个额外的模块，即随机适配器和提示池，我们提出了一种名为参数高效通用度量学习（PUMA）的方法来解决不平衡数据分布和对主导分布的偏见等新挑战。此外，我们还编译了一个包含 8 个不同数据集的新通用度量学习基准。相较于特定数据集模型，PUMA 在使用大约 69 倍较少的可训练参数的情况下表现更好。

具有参数高效迁移学习的通用度量学习

Universal Metric Learning with Parameter-Efficient Transfer Learning

Long-tailed visual recognition has received increasing attention in recent
years. Due to the extremely imbalanced data distribution in long-tailed
learning, the learning process shows great uncertainties. For example, the
predictions of different experts on the same image vary remarkably despite the
same training settings. To alleviate the uncertainty, we propose a Nested
Collaborative Learning (NCL++) which tackles the long-tailed learning problem
by a collaborative learning. To be specific, the collaborative learning
consists of two folds, namely inter-expert collaborative learning (InterCL) and
intra-expert collaborative learning (IntraCL). In-terCL learns multiple experts
collaboratively and concurrently, aiming to transfer the knowledge among
different experts. IntraCL is similar to InterCL, but it aims to conduct the
collaborative learning on multiple augmented copies of the same image within
the single expert. To achieve the collaborative learning in long-tailed
learning, the balanced online distillation is proposed to force the consistent
predictions among different experts and augmented copies, which reduces the
learning uncertainties. Moreover, in order to improve the meticulous
distinguishing ability on the confusing categories, we further propose a Hard
Category Mining (HCM), which selects the negative categories with high
predicted scores as the hard categories. Then, the collaborative learning is
formulated in a nested way, in which the learning is conducted on not just all
categories from a full perspective but some hard categories from a partial
perspective. Extensive experiments manifest the superiority of our method with
outperforming the state-of-the-art whether with using a single model or an
ensemble. The code will be publicly released.

该研究提出了一种名为 Nested Collaborative Learning 的方法，它使用协作学习和平衡的在线蒸馏来解决长尾学习问题，并明显优于现有方法。