A key challenge in dysarthric speech recognition is the speaker-level
diversity attributed to both speaker-identity associated factors such as
gender, and speech impairment severity. Most prior researches on addressing
this issue focused on using speaker-identity only. To this end, this paper
proposes a novel set of techniques to use both severity and speaker-identity in
dysarthric speech recognition: a) multitask training incorporating severity
prediction error; b) speaker-severity aware auxiliary feature adaptation; and
c) structured LHUC transforms separately conditioned on speaker-identity and
severity. Experiments conducted on UASpeech suggest incorporating additional
speech impairment severity into state-of-the-art hybrid DNN, E2E Conformer and
pre-trained Wav2vec 2.0 ASR systems produced statistically significant WER
reductions up to 4.78% (14.03% relative). Using the best system the lowest
published WER of 17.82% (51.25% on very low intelligibility) was obtained on
UASpeech.

本文提出了一套新的方法，即使用多任务训练，包含严重程度预测误差、说话者 - 严重程度感知辅助特征适应以及基于与严重度有关和说话者身份有关的 LHUC 转换来进行发音障碍语音识别，结果实验表明将额外的语音障碍严重度纳入最先进的混合 DNN、E2E Conformer 和预训练的 Wav2vec 2.0 ASR 系统，可以产生显著的 WER 下降，最低 WER 达到 17.82％。

利用言语障碍严重程度进行口吃性言语识别

Use of Speech Impairment Severity for Dysarthric Speech Recognition

Dysarthric speech recognition has posed major challenges due to lack of
training data and heavy mismatch in speaker characteristics. Recent ASR systems
have benefited from readily available pretrained models such as wav2vec2 to
improve the recognition performance. Speaker adaptation using fMLLR and
xvectors have provided major gains for dysarthric speech with very little
adaptation data. However, integration of wav2vec2 with fMLLR features or
xvectors during wav2vec2 finetuning is yet to be explored. In this work, we
propose a simple adaptation network for fine-tuning wav2vec2 using fMLLR
features. The adaptation network is also flexible to handle other speaker
adaptive features such as xvectors. Experimental analysis show steady
improvements using our proposed approach across all impairment severity levels
and attains 57.72\% WER for high severity in UASpeech dataset. We also
performed experiments on German dataset to substantiate the consistency of our
proposed approach across diverse domains.

本研究通过 fine-tuning wav2vec2 并使用 fMLLR 特征进行适应，对发音紊乱的语音识别进行了改进，实验结果表明在不同严重程度的口吃障碍情况下，该方法的识别效果相对稳定，WER 达到了 57.72%。

基于 Wav2vec2 的口齿不清语音识别中的发音人自适应

Speaker adaptation for Wav2vec2 based dysarthric ASR

This work addresses the mismatch problem between the distribution of training
data (source) and testing data (target), in the challenging context of
dysarthric speech recognition. We focus on Speaker Adaptation (SA) in command
speech recognition, where data from multiple sources (i.e., multiple speakers)
are available. Specifically, we propose an unsupervised Multi-Source Domain
Adaptation (MSDA) algorithm based on optimal-transport, called MSDA via
Weighted Joint Optimal Transport (MSDA-WJDOT). We achieve a Command Error Rate
relative reduction of 16% and 7% over the speaker-independent model and the
best competitor method, respectively. The strength of the proposed approach is
that, differently from any other existing SA method, it offers an interpretable
model that can also be exploited, in this context, to diagnose dysarthria
without any specific training. Indeed, it provides a closeness measure between
the target and the source speakers, reflecting their similarity in terms of
speech characteristics. Based on the similarity between the target speaker and
the healthy/dysarthric source speakers, we then define the healthy/dysarthric
score of the target speaker that we leverage to perform dysarthria detection.
This approach does not require any additional training and achieves a 95%
accuracy in the dysarthria diagnosis.

本文提出一种基于最优传输的无监督多源域自适应算法 ——MSDA-WJDOT，用于解决难以识别的发音障碍语音命令识别中通过对多个说话人的数据进行训练以进行说话人自适应的特定问题。该方法相对于其他现有的 SA 方法具有可解释性模型，可用于诊断发音障碍，不需要任何专门的培训，并在发音障碍的诊断中取得 95% 的准确率。