Automatic pronunciation error detection (APED) plays an important role in the
domain of language learning. As for the previous ASR-based APED methods, the
decoded results need to be aligned with the target text so that the errors can
be found out. However, since the decoding process and the alignment process are
independent, the prior knowledge about the target text is not fully utilized.
In this paper, we propose to use the target text as an extra condition for the
Transformer backbone to handle the APED task. The proposed method can output
the error states with consideration of the relationship between the input
speech and the target text in a fully end-to-end fashion.Meanwhile, as the
prior target text is used as a condition for the decoder input, the Transformer
works in a feed-forward manner instead of autoregressive in the inference
stage, which can significantly boost the speed in the actual deployment. We set
the ASR-based Transformer as the baseline APED model and conduct several
experiments on the L2-Arctic dataset. The results demonstrate that our approach
can obtain 8.4\% relative improvement on the $F_1$ score metric.

本文提出了一种使用目标文本作为条件的 Transformer 模型，以在完全端到端的方式下输出考虑输入语音与目标文本关系的错误状态，提高了速度，并在 L2-Arctic 数据集上实验表明相对于 ASR-based 模型，提高了 8.4％的 $ F_1 $ 分数指标。

面向文本的 Transformer 模型用于自动语音发音错误检测

Text-Conditioned Transformer for Automatic Pronunciation Error Detection

Mispronunciation detection and diagnosis (MDD) is a core component of
computer-assisted pronunciation training (CAPT). Most of the existing MDD
approaches focus on dealing with categorical errors (viz. one canonical phone
is substituted by another one, aside from those mispronunciations caused by
deletions or insertions). However, accurate detection and diagnosis of
non-categorial or distortion errors (viz. approximating L2 phones with L1
(first-language) phones, or erroneous pronunciations in between) still seems
out of reach. In view of this, we propose to conduct MDD with a novel end-
to-end automatic speech recognition (E2E-based ASR) approach. In particular, we
expand the original L2 phone set with their corresponding anti-phone set,
making the E2E-based MDD approach have a better capability to take in both
categorical and non-categorial mispronunciations, aiming to provide better
mispronunciation detection and diagnosis feedback. Furthermore, a novel
transfer-learning paradigm is devised to obtain the initial model estimate of
the E2E-based MDD system without resource to any phonological rules. Extensive
sets of experimental results on the L2-ARCTIC dataset show that our best system
can outperform the existing E2E baseline system and pronunciation scoring based
method (GOP) in terms of the F1-score, by 11.05% and 27.71%, respectively.

本论文提出了一种新的端到端自动语音识别方法，使用扩展后的原始 L2 语音集和其对应的反语音集，以更好地识别分类和非分类发音错误，并通过转移学习范式来获得 E2E-based MDD 系统的初始模型估计，在 L2-ARCTIC 数据集上进行了广泛的实验，结果表明我们的最佳系统在 F1 分数方面胜过现有的 E2E 基线系统和基于发音评分 (GOP) 的方法，分别提高了 11.05% 和 27.71%。