Recently, end-to-end (E2E) automatic speech recognition (ASR) systems have
garnered tremendous attention because of their great success and unified
modeling paradigms in comparison to conventional hybrid DNN-HMM ASR systems.
Despite the widespread adoption of E2E modeling frameworks on ASR, there still
is a dearth of work on investigating the E2E frameworks for use in
computer-assisted pronunciation learning (CAPT), particularly for
Mispronunciation detection (MD). In response, we first present a novel use of
hybrid CTCAttention approach to the MD task, taking advantage of the strengths
of both CTC and the attention-based model meanwhile getting around the need for
phone-level forced alignment. Second, we perform input augmentation with text
prompt information to make the resulting E2E model more tailored for the MD
task. On the other hand, we adopt two MD decision methods so as to better
cooperate with the proposed framework: 1) decision-making based on a
recognition confidence measure or 2) simply based on speech recognition
results. A series of Mandarin MD experiments demonstrate that our approach not
only simplifies the processing pipeline of existing hybrid DNN-HMM systems but
also brings about systematic and substantial performance improvements.
Furthermore, input augmentation with text prompts seems to hold excellent
promise for the E2E-based MD approach.

本研究采用 E2E 模型中的混合 CTC-Attention 方法，并针对中文发音检测任务进行了输入扩充，使结果更加适合该任务。实验结果表明，与传统的混合 DNN-HMM 系统相比，该方法能够大大简化处理流程并显著提高性能。