Confidence estimation, in which we estimate the reliability of each
recognized token (e.g., word, sub-word, and character) in automatic speech
recognition (ASR) hypotheses and detect incorrectly recognized tokens, is an
important function for developing ASR applications. In this study, we perform
confidence estimation for end-to-end (E2E) ASR hypotheses. Recent E2E ASR
systems show high performance (e.g., around 5% token error rates) for various
ASR tasks. In such situations, confidence estimation becomes difficult since we
need to detect infrequent incorrect tokens from mostly correct token sequences.
To tackle this imbalanced dataset problem, we employ a bidirectional long
short-term memory (BLSTM)-based model as a strong binary-class
(correct/incorrect) sequence labeler that is trained with a class balancing
objective. We experimentally confirmed that, by utilizing several types of ASR
decoding scores as its auxiliary features, the model steadily shows high
confidence estimation performance under highly imbalanced settings. We also
confirmed that the BLSTM-based model outperforms Transformer-based confidence
estimation models, which greatly underestimate incorrect tokens.

本研究利用双向长短时记忆（BLSTM）模型进行自动语音识别（ASR）中的可信度估计，通过利用多种 ASR 解码得分作为辅助特征，该模型在高度不平衡的情况下稳定展现出高可信度估计表现，并且大幅优于基于 Transformer 的可信度估计模型。