Disordered speech recognition profound implications for improving the quality
of life for individuals afflicted with, for example, dysarthria. Dysarthric
speech recognition encounters challenges including limited data, substantial
dissimilarities between dysarthric and non-dysarthric speakers, and significant
speaker variations stemming from the disorder. This paper introduces
Perceiver-Prompt, a method for speaker adaptation that utilizes P-Tuning on the
Whisper large-scale model. We first fine-tune Whisper using LoRA and then
integrate a trainable Perceiver to generate fixed-length speaker prompts from
variable-length inputs, to improve model recognition of Chinese dysarthric
speech. Experimental results from our Chinese dysarthric speech dataset
demonstrate consistent improvements in recognition performance with
Perceiver-Prompt. Relative reduction up to 13.04% in CER is obtained over the
fine-tuned Whisper.

通过引入 Perceiver-Prompt 方法，利用 P-Tuning 对 Whisper 大规模模型进行微调，并通过可训练的 Perceiver 从可变长度输入中生成固定长度的说话人提示，以提高对中国发音障碍语音的模型识别性能。我们的实验结果表明，Perceiver-Prompt 在中国发音障碍语音数据集中获得了持续的识别性能改善，CER 相对减少高达 13.04%。