Recent advances in deep learning and automatic speech recognition (ASR) have
enabled the end-to-end (E2E) ASR system and boosted the accuracy to a new
level. The E2E systems implicitly model all conventional ASR components, such
as the acoustic model (AM) and the language model (LM), in a single network
trained on audio-text pairs. Despite this simpler system architecture, fusing a
separate LM, trained exclusively on text corpora, into the E2E system has
proven to be beneficial. However, the application of LM fusion presents certain
drawbacks, such as its inability to address the domain mismatch issue inherent
to the internal AM. Drawing inspiration from the concept of LM fusion, we
propose the integration of an external AM into the E2E system to better address
the domain mismatch. By implementing this novel approach, we have achieved a
significant reduction in the word error rate, with an impressive drop of up to
14.3% across varied test sets. We also discovered that this AM fusion approach
is particularly beneficial in enhancing named entity recognition.

最近在深度学习和自动语音识别（ASR）方面的进展使得端到端（E2E）ASR 系统成为可能，并且将准确性提升到一个新的水平。我们提出将外部的 AM 整合到 E2E 系统中，以更好地解决领域不匹配问题，并取得了显著的词错误率降低，尤其在增强命名实体识别方面效果明显。

终端到终端音频识别的声学模型融合

Acoustic Model Fusion for End-to-end Speech Recognition

ASR model deployment environment is ever-changing, and the incoming speech
can be switched across different domains during a session. This brings a
challenge for effective domain adaptation when only target domain text data is
available, and our objective is to obtain obviously improved performance on the
target domain while the performance on the general domain is less undermined.
In this paper, we propose an adaptive LM fusion approach called internal
language model estimation based adaptive domain adaptation (ILME-ADA). To
realize such an ILME-ADA, an interpolated log-likelihood score is calculated
based on the maximum of the scores from the internal LM and the external LM
(ELM) respectively. We demonstrate the efficacy of the proposed ILME-ADA method
with both RNN-T and LAS modeling frameworks employing neural network and n-gram
LMs as ELMs respectively on two domain specific (target) test sets. The
proposed method can achieve significantly better performance on the target test
sets while it gets minimal performance degradation on the general test set,
compared with both shallow and ILME-based LM fusion methods.

本文提出了一种基于内部语言模型估计的自适应领域适应方法，通过最大化内部和外部语言模型 (ELM) 的分数的插值对数似然分数来实现 ILME-ADA，该方法在 RNN-T 和 LAS 模型框架下使用神经网络和 n-gram 语言模型作为 ELM，在两个特定领域 (目标) 测试集上表现显著优于浅层和基于 ILME 的 LM 融合方法，且在一般测试集上性能下降最小。