End-to-end (E2E) models have made rapid progress in automatic speech
recognition (ASR) and perform competitively relative to conventional models. To
further improve the quality, a two-pass model has been proposed to rescore
streamed hypotheses using the non-streaming Listen, Attend and Spell (LAS)
model while maintaining a reasonable latency. The model attends to acoustics to
rescore hypotheses, as opposed to a class of neural correction models that use
only first-pass text hypotheses. In this work, we propose to attend to both
acoustics and first-pass hypotheses using a deliberation network. A
bidirectional encoder is used to extract context information from first-pass
hypotheses. The proposed deliberation model achieves 12% relative WER reduction
compared to LAS rescoring in Google Voice Search (VS) tasks, and 23% reduction
on a proper noun test set. Compared to a large conventional model, our best
model performs 21% relatively better for VS. In terms of computational
complexity, the deliberation decoder has a larger size than the LAS decoder,
and hence requires more computations in second-pass decoding.

本篇文章探讨了使用 deliberation network 既关注声学特征又关注第一步文本假说的方法来提高两步叠加模型在 ASR 中性能的优化，经过 Google Voice Search 的比较实验，正确率提高了 12%（相对于 LAS rescoring），在专有名词测试集上，提高了 23%。与传统大模型相比，最好的模型在 VS 上表现要好 21%。