The research community has long studied computer-assisted pronunciation
training (CAPT) methods in non-native speech. Researchers focused on studying
various model architectures, such as Bayesian networks and deep learning
methods, as well as on the analysis of different representations of the speech
signal. Despite significant progress in recent years, existing CAPT methods are
not able to detect pronunciation errors with high accuracy (only 60\% precision
at 40\%-80\% recall). One of the key problems is the low availability of
mispronounced speech that is needed for the reliable training of pronunciation
error detection models. If we had a generative model that could mimic
non-native speech and produce any amount of training data, then the task of
detecting pronunciation errors would be much easier. We present three
innovative techniques based on phoneme-to-phoneme (P2P), text-to-speech (T2S),
and speech-to-speech (S2S) conversion to generate correctly pronounced and
mispronounced synthetic speech. We show that these techniques not only improve
the accuracy of three machine learning models for detecting pronunciation
errors but also help establish a new state-of-the-art in the field. Earlier
studies have used simple speech generation techniques such as P2P conversion,
but only as an additional mechanism to improve the accuracy of pronunciation
error detection. We, on the other hand, consider speech generation to be the
first-class method of detecting pronunciation errors. The effectiveness of
these techniques is assessed in the tasks of detecting pronunciation and
lexical stress errors. Non-native English speech corpora of German, Italian,
and Polish speakers are used in the evaluations. The best proposed S2S
technique improves the accuracy of detecting pronunciation errors in AUC metric
by 41\% from 0.528 to 0.749 compared to the state-of-the-art approach.

本研究提出了基于音素到音素转换、文本到语音转换和语音到语音转换三个方法来生成正确发音和发音错误的合成语音，将语音生成作为检测语音发音错误的一流方法，并在检测发音和词汇强调错误的任务中评估了这些技术，证明这些技术不仅提高了三个机器学习模型检测发音错误的准确性，而且还有助于建立该领域的新的技术标准。

计算机辅助发音训练 -- 语音合成几乎是您所需的全部

Computer-assisted Pronunciation Training -- Speech synthesis is almost  all you need

Mispronunciation detection and diagnosis (MDD) is a core component of
computer-assisted pronunciation training (CAPT). Most of the existing MDD
approaches focus on dealing with categorical errors (viz. one canonical phone
is substituted by another one, aside from those mispronunciations caused by
deletions or insertions). However, accurate detection and diagnosis of
non-categorial or distortion errors (viz. approximating L2 phones with L1
(first-language) phones, or erroneous pronunciations in between) still seems
out of reach. In view of this, we propose to conduct MDD with a novel end-
to-end automatic speech recognition (E2E-based ASR) approach. In particular, we
expand the original L2 phone set with their corresponding anti-phone set,
making the E2E-based MDD approach have a better capability to take in both
categorical and non-categorial mispronunciations, aiming to provide better
mispronunciation detection and diagnosis feedback. Furthermore, a novel
transfer-learning paradigm is devised to obtain the initial model estimate of
the E2E-based MDD system without resource to any phonological rules. Extensive
sets of experimental results on the L2-ARCTIC dataset show that our best system
can outperform the existing E2E baseline system and pronunciation scoring based
method (GOP) in terms of the F1-score, by 11.05% and 27.71%, respectively.

本论文提出了一种新的端到端自动语音识别方法，使用扩展后的原始 L2 语音集和其对应的反语音集，以更好地识别分类和非分类发音错误，并通过转移学习范式来获得 E2E-based MDD 系统的初始模型估计，在 L2-ARCTIC 数据集上进行了广泛的实验，结果表明我们的最佳系统在 F1 分数方面胜过现有的 E2E 基线系统和基于发音评分 (GOP) 的方法，分别提高了 11.05% 和 27.71%。