Accurately detecting dysfluencies in spoken language can help to improve the
performance of automatic speech and language processing components and support
the development of more inclusive speech and language technologies. Inspired by
the recent trend towards the deployment of large language models (LLMs) as
universal learners and processors of non-lexical inputs, such as audio and
video, we approach the task of multi-label dysfluency detection as a language
modeling problem. We present hypotheses candidates generated with an automatic
speech recognition system and acoustic representations extracted from an audio
encoder model to an LLM, and finetune the system to predict dysfluency labels
on three datasets containing English and German stuttered speech. The
experimental results show that our system effectively combines acoustic and
lexical information and achieves competitive results on the multi-label
stuttering detection task.

通过将自动语音识别系统生成的假设候选项和从音频编码模型提取的声学表示输入到大型语言模型（LLMs）中，我们将多标签异味检测任务作为一种语言建模问题进行了研究，并在包含英语和德语结巴语音的三个数据集上对系统进行了优化，以预测异味标签，实验结果表明我们的系统有效地结合声学和词法信息，在多标签结巴检测任务上取得了有竞争力的结果。

大型语言模型用于口吃语音的错断检测

Large Language Models for Dysfluency Detection in Stuttered Speech

The adoption of advanced deep learning architectures in stuttering detection
(SD) tasks is challenging due to the limited size of the available datasets. To
this end, this work introduces the application of speech embeddings extracted
from pre-trained deep learning models trained on large audio datasets for
different tasks. In particular, we explore audio representations obtained using
emphasized channel attention, propagation, and aggregation time delay neural
network (ECAPA-TDNN) and Wav2Vec2.0 models trained on VoxCeleb and LibriSpeech
datasets respectively. After extracting the embeddings, we benchmark with
several traditional classifiers, such as the K-nearest neighbour (KNN),
Gaussian naive Bayes, and neural network, for the SD tasks. In comparison to
the standard SD systems trained only on the limited SEP-28k dataset, we obtain
a relative improvement of 12.08%, 28.71%, 37.9% in terms of unweighted average
recall (UAR) over the baselines. Finally, we have shown that combining two
embeddings and concatenating multiple layers of Wav2Vec2.0 can further improve
the UAR by up to 2.60% and 6.32% respectively.

本研究通过使用预先训练的深度学习模型提取的语音嵌入，探索了使用 ECAPA-TDNN 和 Wav2Vec2.0 模型进行口吃检测任务的音频表示。相对于仅在有限 SEP-28k 数据集上训练的标准口吃检测系统，本研究在多个传统分类器上获得了 12.08％、28.71％、37.9％的相对改进并进一步证明结合两个嵌入和连接多个层的 Wav2Vec2.0 可进一步提高其性能。

使用说话人表示和自监督上下文嵌入进行口吃检测

Stuttering Detection Using Speaker Representations and Self-supervised  Contextual Embeddings

In this paper, we present end-to-end and speech embedding based systems
trained in a self-supervised fashion to participate in the ACM Multimedia 2022
ComParE Challenge, specifically the stuttering sub-challenge. In particular, we
exploit the embeddings from the pre-trained Wav2Vec2.0 model for stuttering
detection (SD) on the KSoF dataset. After embedding extraction, we benchmark
with several methods for SD. Our proposed self-supervised based SD system
achieves a UAR of 36.9% and 41.0% on validation and test sets respectively,
which is 31.32% (validation set) and 1.49% (test set) higher than the best
(DeepSpectrum) challenge baseline (CBL). Moreover, we show that concatenating
layer embeddings with Mel-frequency cepstral coefficients (MFCCs) features
further improves the UAR of 33.81% and 5.45% on validation and test sets
respectively over the CBL. Finally, we demonstrate that the summing information
across all the layers of Wav2Vec2.0 surpasses the CBL by a relative margin of
45.91% and 5.69% on validation and test sets respectively. Grand-challenge:
Computational Paralinguistics ChallengE

本文提出了基于自监督学习的语音嵌入系统，通过对预先训练的 Wav2Vec2.0 模型进行嵌入提取，结合 Mel 频率倒谱系数 (MFCC) 特征进行评估，在计算语言学竞赛中达到了较好的结果，相对于 DeepSpectrum 挑战基线提高了 31.32% (验证集) 和 1.49% (测试集)。同时，通过对 Wav2Vec2.0 的各层嵌入进行求和，进一步提高了系统性能。