A key challenge in dysarthric speech recognition is the speaker-level
diversity attributed to both speaker-identity associated factors such as
gender, and speech impairment severity. Most prior researches on addressing
this issue focused on using speaker-identity only. To this end, this paper
proposes a novel set of techniques to use both severity and speaker-identity in
dysarthric speech recognition: a) multitask training incorporating severity
prediction error; b) speaker-severity aware auxiliary feature adaptation; and
c) structured LHUC transforms separately conditioned on speaker-identity and
severity. Experiments conducted on UASpeech suggest incorporating additional
speech impairment severity into state-of-the-art hybrid DNN, E2E Conformer and
pre-trained Wav2vec 2.0 ASR systems produced statistically significant WER
reductions up to 4.78% (14.03% relative). Using the best system the lowest
published WER of 17.82% (51.25% on very low intelligibility) was obtained on
UASpeech.

本文提出了一套新的方法，即使用多任务训练，包含严重程度预测误差、说话者 - 严重程度感知辅助特征适应以及基于与严重度有关和说话者身份有关的 LHUC 转换来进行发音障碍语音识别，结果实验表明将额外的语音障碍严重度纳入最先进的混合 DNN、E2E Conformer 和预训练的 Wav2vec 2.0 ASR 系统，可以产生显著的 WER 下降，最低 WER 达到 17.82％。