Recent advances in large language models (LLMs) have promoted generative
error correction (GER) for automatic speech recognition (ASR), which aims to
predict the ground-truth transcription from the decoded N-best hypotheses.
Thanks to the strong language generation ability of LLMs and rich information
in the N-best list, GER shows great effectiveness in enhancing ASR results.
However, it still suffers from two limitations: 1) LLMs are unaware of the
source speech during GER, which may lead to results that are grammatically
correct but violate the source speech content, 2) N-best hypotheses usually
only vary in a few tokens, making it redundant to send all of them for GER,
which could confuse LLM about which tokens to focus on and thus lead to
increased miscorrection. In this paper, we propose ClozeGER, a new paradigm for
ASR generative error correction. First, we introduce a multimodal LLM (i.e.,
SpeechGPT) to receive source speech as extra input to improve the fidelity of
correction output. Then, we reformat GER as a cloze test with logits
calibration to remove the input information redundancy and simplify GER with
clear instructions. Experiments show that ClozeGER achieves a new breakthrough
over vanilla GER on 9 popular ASR datasets.

该论文提出了一种新的 ASR 生成性错误纠正范式 ClozeGER，通过引入一种多模态 LLM（即 SpeechGPT）来改善纠正输出的忠实度，然后将 GER 重新设计为带有 logits 校准的 cloze 测试，以消除输入信息冗余并简化 GER 过程。实验证明，ClozeGER 在 9 个流行的 ASR 数据集上取得了新的突破。