Text-only and semi-supervised training based on audio-only data has gained
popularity recently due to the wide availability of unlabeled text and speech
data. In this work, we propose incorporating text-only and semi-supervised
training into an attention-based deliberation model. By incorporating text-only
data in training a bidirectional encoder representation from transformer (BERT)
for the deliberation text encoder, and large-scale text-to-speech and
audio-only utterances using joint acoustic and text decoder (JATD) and
semi-supervised training, we achieved 4%-12% WER reduction for various tasks
compared to the baseline deliberation. Compared to a state-of-the-art language
model (LM) rescoring method, the deliberation model reduces the Google Voice
Search WER by 11% relative. We show that the deliberation model also achieves a
positive human side-by-side evaluation compared to the state-of-the-art LM
rescorer with reasonable endpointer latencies.

通过使用文本数据和半监督训练，结合基于 Transformer 的双向编码器表示和文本到语音解码器以及半监督训练，实现了多项任务中 4% 至 12% 的 WER 降低，并比最先进的语言模型（LM）缩减的 Deliberation 模型相比降低了 11% 的 Google Voice 搜索 WER，达到了与最先进的 LM 重评分器相当的人类评估结果。