Text-only adaptation of a transducer model remains challenging for end-to-end
speech recognition since the transducer has no clearly separated acoustic model
(AM), language model (LM) or blank model. In this work, we propose a modular
hybrid autoregressive transducer (MHAT) that has structurally separated label
and blank decoders to predict label and blank distributions, respectively,
along with a shared acoustic encoder. The encoder and label decoder outputs are
directly projected to AM and internal LM scores and then added to compute label
posteriors. We train MHAT with an internal LM loss and a HAT loss to ensure
that its internal LM becomes a standalone neural LM that can be effectively
adapted to text. Moreover, text adaptation of MHAT fosters a much better LM
fusion than internal LM subtraction-based methods. On Google's large-scale
production data, a multi-domain MHAT adapted with 100B sentences achieves
relative WER reductions of up to 12.4% without LM fusion and 21.5% with LM
fusion from 400K-hour trained HAT.

本文提出了一种模块化混合自回归转录器（MHAT），该转录器具有结构上分离的标签和空白解码器，涉及共享的声学编码器，因此在语音识别中具有方便的文本自适应性和内部语言模型。在大量实验中该方法的内部语言模型具有高效性，因此在 Google 大规模生产数据中，使用 MHAT 相比 400K 小时的 HAT 训练，进行文本自适应后，无需语言融合可获得高达 12.4％的相对字错率减少和 21.5％的相对字错率减少与语言融合。

模块化混合自回归传感器

Modular Hybrid Autoregressive Transducer

Language models (LMs) significantly improve the recognition accuracy of
end-to-end (E2E) models on words rarely seen during training, when used in
either the shallow fusion or the rescoring setups. In this work, we introduce
LMs in the learning of hybrid autoregressive transducer (HAT) models in the
discriminative training framework, to mitigate the training versus inference
gap regarding the use of LMs. For the shallow fusion setup, we use LMs during
both hypotheses generation and loss computation, and the LM-aware MWER-trained
model achieves 10\% relative improvement over the model trained with standard
MWER on voice search test sets containing rare words. For the rescoring setup,
we learn a small neural module to generate per-token fusion weights in a
data-dependent manner. This model achieves the same rescoring WER as regular
MWER-trained model, but without the need for sweeping fusion weights.

该研究介绍了在 discriminative training framework 中如何在 hybrid autoregressive transducer (HAT) 模型的学习中引入语言模型 (LMs) 来缓解训练和推理过程中使用 LMs 存在的差距。在浅层融合和重甄时，使用 LMs 显着提高了仅在训练期间极少见的单词的识别准确性。