Speech emotion recognition (SER) plays a crucial role in human-computer
interaction. The emergence of edge devices in the Internet of Things (IoT)
presents challenges in constructing intricate deep learning models due to
constraints in memory and computational resources. Moreover, emotional speech
data often contains private information, raising concerns about privacy leakage
during the deployment of SER models. To address these challenges, we propose a
data distillation framework to facilitate efficient development of SER models
in IoT applications using a synthesised, smaller, and distilled dataset. Our
experiments demonstrate that the distilled dataset can be effectively utilised
to train SER models with fixed initialisation, achieving performances
comparable to those developed using the original full emotional speech dataset.

通过合成、缩小和提炼数据集的方法，我们提出了一种数据蒸馏框架，以促进在物联网应用中高效开发情感语音识别模型。我们的实验表明，可以有效地利用提炼后的数据集来训练情感语音识别模型，并且性能与使用原始完整的情感语音数据集开发的模型相当。

资源约束下基于数据蒸馏的语音情感识别

Speech Emotion Recognition under Resource Constraints with Data  Distillation

Emotion datasets used for Speech Emotion Recognition (SER) often contain
acted or elicited speech, limiting their applicability in real-world scenarios.
In this work, we used the Emotional Voice Messages (EMOVOME) database,
including spontaneous voice messages from conversations of 100 Spanish speakers
on a messaging app, labeled in continuous and discrete emotions by expert and
non-expert annotators. We created speaker-independent SER models using the
eGeMAPS features, transformer-based models and their combination. We compared
the results with reference databases and analyzed the influence of annotators
and gender fairness. The pre-trained Unispeech-L model and its combination with
eGeMAPS achieved the highest results, with 61.64% and 55.57% Unweighted
Accuracy (UA) for 3-class valence and arousal prediction respectively, a 10%
improvement over baseline models. For the emotion categories, 42.58% UA was
obtained. EMOVOME performed lower than the acted RAVDESS database. The elicited
IEMOCAP database also outperformed EMOVOME in the prediction of emotion
categories, while similar results were obtained in valence and arousal.
Additionally, EMOVOME outcomes varied with annotator labels, showing superior
results and better fairness when combining expert and non-expert annotations.
This study significantly contributes to the evaluation of SER models in
real-life situations, advancing in the development of applications for
analyzing spontaneous voice messages.

使用来自西班牙 100 位说话者之间的即兴语音消息的情感数据集（EMOVOME 数据库），利用 eGeMAPS 特征、基于 Transformer 的模型及其组合，创建了基于说话者的独立 SER 模型，对连续和离散情感进行标注，并比较参考数据库的结果，并分析注释者和性别公平性的影响。使用预训练的 Unispeech-L 模型及其与 eGeMAPS 的组合取得了最高的结果，对于 3 类情感价值和唤醒度的预测，分别达到了 61.64% 和 55.57% 的非加权准确率（UA），相比基线模型提高了 10%。对于情感类别，获得了 42.58% 的 UA。EMOVOME 在情感类别的预测上不及被操纵的 RAVDESS 数据库，在情感价值和唤醒度方面与被引发的 IEMOCAP 数据库效果相似。此外，EMOVOME 的结果随着注释者标签的变化而变化，当结合专家和非专家的注释时，表现出更好的结果和更好的公平性。该研究在评估 SER 模型在现实生活中的应用方面做出了重要贡献，推动了用于分析即兴语音消息的应用程序的开发。

野外录制的语音信息中的语音情感识别

Speech emotion recognition from voice messages recorded in the wild

Speech contains rich information on the emotions of humans, and Speech
Emotion Recognition (SER) has been an important topic in the area of
human-computer interaction. The robustness of SER models is crucial,
particularly in privacy-sensitive and reliability-demanding domains like
private healthcare. Recently, the vulnerability of deep neural networks in the
audio domain to adversarial attacks has become a popular area of research.
However, prior works on adversarial attacks in the audio domain primarily rely
on iterative gradient-based techniques, which are time-consuming and prone to
overfitting the specific threat model. Furthermore, the exploration of sparse
perturbations, which have the potential for better stealthiness, remains
limited in the audio domain. To address these challenges, we propose a
generator-based attack method to generate sparse and transferable adversarial
examples to deceive SER models in an end-to-end and efficient manner. We
evaluate our method on two widely-used SER datasets, Database of Elicited Mood
in Speech (DEMoS) and Interactive Emotional dyadic MOtion CAPture (IEMOCAP),
and demonstrate its ability to generate successful sparse adversarial examples
in an efficient manner. Moreover, our generated adversarial examples exhibit
model-agnostic transferability, enabling effective adversarial attacks on
advanced victim models.

通过生成器的攻击方法，我们能够以高效的方式生成稀疏且可转移的对抗性示例，从而欺骗语音情感识别模型，并能够对先进的受害者模型进行有效的对抗攻击。