We propose a new two-pass E2E speech recognition model that improves ASR
performance by training on a combination of paired data and unpaired text data.
Previously, the joint acoustic and text decoder (JATD) has shown promising
results through the use of text data during model training and the recently
introduced deliberation architecture has reduced recognition errors by
leveraging first-pass decoding results. Our method, dubbed Deliberation-JATD,
combines the spelling correcting abilities of deliberation with JATD's use of
unpaired text data to further improve performance. The proposed model produces
substantial gains across multiple test sets, especially those focused on rare
words, where it reduces word error rate (WER) by between 12% and 22.5%
relative. This is done without increasing model size or requiring multi-stage
training, making Deliberation-JATD an efficient candidate for on-device
applications.

提出一种新的两阶段端到端的语音识别模型，通过训练配对数据和未配对文本数据的组合来提高 ASR 性能，称为 Deliberation-JATD，它结合了 Deliberation 的拼写校正能力和 JATD 的未配对文本数据使用，实现了在多个测试集上的显著提高，特别是在稀有词汇方面，相对降低了词误率 12% 至 22.5%。同时该模型不需要增加模型大小或多阶段训练，成为一种高效的适用于设备的候选模型。