Language models (LMs) have long been used to improve results of automatic
speech recognition (ASR) systems, but they are unaware of the errors that ASR
systems make. Error correction models are designed to fix ASR errors, however,
they showed little improvement over traditional LMs mainly due to the lack of
supervised training data. In this paper, we present Denoising LM (DLM), which
is a $\textit{scaled}$ error correction model trained with vast amounts of
synthetic data, significantly exceeding prior attempts meanwhile achieving new
state-of-the-art ASR performance. We use text-to-speech (TTS) systems to
synthesize audio, which is fed into an ASR system to produce noisy hypotheses,
which are then paired with the original texts to train the DLM. DLM has several
$\textit{key ingredients}$: (i) up-scaled model and data; (ii) usage of
multi-speaker TTS systems; (iii) combination of multiple noise augmentation
strategies; and (iv) new decoding techniques. With a Transformer-CTC ASR, DLM
achieves 1.5% word error rate (WER) on $\textit{test-clean}$ and 3.3% WER on
$\textit{test-other}$ on Librispeech, which to our knowledge are the best
reported numbers in the setting where no external audio data are used and even
match self-supervised methods which use external audio data. Furthermore, a
single DLM is applicable to different ASRs, and greatly surpassing the
performance of conventional LM based beam-search rescoring. These results
indicate that properly investigated error correction models have the potential
to replace conventional LMs, holding the key to a new level of accuracy in ASR
systems.

使用大量合成数据进行训练的缩放误差校正模型 (Denoising LM) 在自动语音识别 (ASR) 系统中实现了最先进的性能表现，通过合成语音进入 ASR 系统并与原始文本配对训练，DLM 取得了 1.5% 的单词错误率 (WER) 以及在 Librispeech 上新的 ASR 性能记录，并展示了取代传统语言模型的潜力。