The automated classification of stuttered speech has significant implications
for timely assessments providing assistance to speech language pathologists.
Despite notable advancements in the field, the cases in which multiple
disfluencies occur in speech require attention. We have taken a progressive
approach to fill this gap by classifying multi-stuttered speech more
efficiently. The problem has been addressed by firstly curating a dataset of
multi-stuttered disfluencies from SEP-28k audio clips. Secondly, employing
Whisper, a state-of-the-art speech recognition model has been leveraged by
using its encoder and taking the problem as multi-label classification.
Thirdly, using a 6 encoder layer Whisper and experimenting with various layer
freezing strategies, a computationally efficient configuration of the model was
identified. The proposed configuration achieved micro, macro, and weighted F1-
scores of 0.88, 0.85, and 0.87, correspondingly on an external test dataset
i.e. Fluency-Bank. In addition, through layer freezing strategies, we were able
to achieve the aforementioned results by fine-tuning a single encoder layer,
consequently, reducing the model's trainable parameters from 20.27 million to
3.29 million. This research study unveils the contribution of the last encoder
layer in the identification of disfluencies in stuttered speech. Consequently,
it has led to a computationally efficient approach which makes the model more
adaptable for various dialects and languages.

通过使用先进的语音识别模型 Whisper 以及 6 层编码器的不同层冻结策略，本研究提出了一种计算高效的模型配置来更有效地对多发性口吃语音进行自动分类，取得了显著的性能，同时发现最后一个编码器层在辨别口吃性不流畅语音方面的贡献，使模型适用于多种方言和语言。

优化多重口吃语音分类：利用 Whisper 的编码器实现自动评估中高效参数减少

Optimizing Multi-Stuttered Speech Classification: Leveraging Whisper's  Encoder for Efficient Parameter Reduction in Automated Assessment

In recent years, advancements in the field of speech processing have led to
cutting-edge deep learning algorithms with immense potential for real-world
applications. The automated identification of stuttered speech is one of such
applications that the researchers are addressing by employing deep learning
techniques. Recently, researchers have utilized Wav2vec2.0, a speech
recognition model to classify disfluency types in stuttered speech. Although
Wav2vec2.0 has shown commendable results, its ability to generalize across all
disfluency types is limited. In addition, since its base model uses 12 encoder
layers, it is considered a resource-intensive model. Our study unravels the
capabilities of Whisper for the classification of disfluency types in stuttered
speech. We have made notable contributions in three pivotal areas: enhancing
the quality of SEP28-k benchmark dataset, exploration of Whisper for
classification, and introducing an efficient encoder layer freezing strategy.
The optimized Whisper model has achieved the average F1-score of 0.81, which
proffers its abilities. This study also unwinds the significance of deeper
encoder layers in the identification of disfluency types, as the results
demonstrate their greater contribution compared to initial layers. This
research represents substantial contributions, shifting the emphasis towards an
efficient solution, thereby thriving towards prospective innovation.

近年来，在语音处理领域的进展已经引发出具有巨大潜力的尖端深度学习算法，而自动识别口吃症言语是这些研究者采用深度学习技术所致力解决的应用之一。本研究通过旨在聚焦于高效解决方案的重要贡献，解开了 Whisper 在口吃症言语类型分类上的能力，并对较深层编码器的重要性进行了探究。