Automatic Speech Recognition (ASR) technology has made significant progress
in recent years, providing accurate transcription across various domains.
However, some challenges remain, especially in noisy environments and
specialized jargon. In this paper, we propose a novel approach for improved
jargon word recognition by contextual biasing Whisper-based models. We employ a
keyword spotting model that leverages the Whisper encoder representation to
dynamically generate prompts for guiding the decoder during the transcription
process. We introduce two approaches to effectively steer the decoder towards
these prompts: KG-Whisper, which is aimed at fine-tuning the Whisper decoder,
and KG-Whisper-PT, which learns a prompt prefix. Our results show a significant
improvement in the recognition accuracy of specified keywords and in reducing
the overall word error rates. Specifically, in unseen language generalization,
we demonstrate an average WER improvement of 5.1% over Whisper.

通过上下文偏差对 Whisper 模型进行改进，提出了一种优化行话词识别的新方法。采用关键词检测模型，利用 Whisper 编码器表示动态生成的提示来引导解码器。引入了 KG-Whisper 和 KG-Whisper-PT 两种方法来有效引导解码器，并在特定关键词的识别准确率和整体词错误率上取得了显著的改进。在未见过的语言泛化中，与 Whisper 相比，平均词错误率提高了 5.1%。