Speech contains information that is clinically relevant to some diseases,
which has the potential to be used for health assessment. Recent work shows an
interest in applying deep learning algorithms, especially pretrained large
speech models to the applications of Automatic Speech Assessment. One question
that has not been explored is how these models output the results based on
their inputs. In this work, we train and compare two configurations of Audio
Spectrogram Transformer in the context of Voice Disorder Detection and apply
the attention rollout method to produce model relevance maps, the computed
relevance of the spectrogram regions when the model makes predictions. We use
these maps to analyse how models make predictions in different conditions and
to show that the spread of attention is reduced as a model is finetuned, and
the model attention is concentrated on specific phoneme regions.

训练和比较两种配置下的音频频谱变换器，用于语音障碍检测；应用注意力回传方法生成模型相关性图，分析模型在不同条件下的预测方式，证明随着模型微调，注意力的扩散减少，重点集中在特定音素区域。

自然语言预训练模型在自动语音评估语音障碍中的解释

Interpreting Pretrained Speech Models for Automatic Speech Assessment of  Voice Disorders

Automatic Speech Assessment (ASA) has seen notable advancements with the
utilization of self-supervised features (SSL) in recent research. However, a
key challenge in ASA lies in the imbalanced distribution of data, particularly
evident in English test datasets. To address this challenge, we approach ASA as
an ordinal classification task, introducing Weighted Vectors Ranking Similarity
(W-RankSim) as a novel regularization technique. W-RankSim encourages closer
proximity of weighted vectors in the output layer for similar classes, implying
that feature vectors with similar labels would be gradually nudged closer to
each other as they converge towards corresponding weighted vectors. Extensive
experimental evaluations confirm the effectiveness of our approach in improving
ordinal classification performance for ASA. Furthermore, we propose a hybrid
model that combines SSL and handcrafted features, showcasing how the inclusion
of handcrafted features enhances performance in an ASA system.

自动语音评估（ASA）通过最近的研究中使用自监督特征（SSL）取得了显著进展。然而，在 ASA 中存在一个关键挑战，即数据的不平衡分布，特别在英语测试数据集中表现明显。为了解决这个挑战，我们将 ASA 视为一个序列分类任务，引入了加权向量排序相似度（W-RankSim）作为一种新的正则化技术。W-RankSim 鼓励输出层中加权向量更靠近类似类别，这意味着具有相似标签的特征向量在收敛到相应的加权向量时逐渐靠近彼此。广泛的实验评估确认了我们方法在提高 ASA 的序列分类性能方面的有效性。此外，我们提出了一个混合模型，结合了 SSL 和手工制作的特征，展示了手工制作特征如何增强 ASA 系统的性能。