Recently, the usage of speech self-supervised models (SSL) for downstream
tasks has been drawing a lot of attention. While large pre-trained models
commonly outperform smaller models trained from scratch, questions regarding
the optimal fine-tuning strategies remain prevalent. In this paper, we explore
the fine-tuning strategies of the WavLM Large model for the speech emotion
recognition task on the MSP Podcast Corpus. More specifically, we perform a
series of experiments focusing on using gender and semantic information from
utterances. We then sum up our findings and describe the final model we used
for submission to Speech Emotion Recognition Challenge 2024.

最近，人们对语音自我监督模型（SSL）在下游任务中的使用越来越关注。在大规模预训练模型通常优于从头开始训练的较小模型的情况下，人们对最佳微调策略的问题仍然很普遍。本文中，我们探讨了 WavLM Large 模型在 MSP Podcast Corpus 的语音情感识别任务中的微调策略。具体而言，我们进行了一系列实验，重点研究了来自话语中的性别和语义信息的使用。然后，我们总结了我们的发现，并描述了我们用于提交 Speech Emotion Recognition Challenge 2024 的最终模型。

将 WavLM 应用于语音情感识别

Adapting WavLM for Speech Emotion Recognition

Spoken language understanding (SLU) is a task aiming to extract high-level
semantics from spoken utterances. Previous works have investigated the use of
speech self-supervised models and textual pre-trained models, which have shown
reasonable improvements to various SLU tasks. However, because of the
mismatched modalities between speech signals and text tokens, previous methods
usually need complex designs of the frameworks. This work proposes a simple yet
efficient unsupervised paradigm that connects speech and textual pre-trained
models, resulting in an unsupervised speech-to-semantic pre-trained model for
various tasks in SLU. To be specific, we propose to use unsupervised automatic
speech recognition (ASR) as a connector that bridges different modalities used
in speech and textual pre-trained models. Our experiments show that
unsupervised ASR itself can improve the representations from speech
self-supervised models. More importantly, it is shown as an efficient connector
between speech and textual pre-trained models, improving the performances of
five different SLU tasks. Notably, on spoken question answering, we reach the
state-of-the-art result over the challenging NMSQA benchmark.

该研究提出了一种简单高效的无监督语音到语义预训练模型，使用无监督自动语音识别作为连接器，实现了不同任务的语音语言理解，其中包括最近热门的口语问答任务，并在 NMSQA 基准测试中达到了最新的最优结果。