The phonological discrepancies between a speaker's native (L1) and the
non-native language (L2) serves as a major factor for mispronunciation. This
paper introduces a novel multilingual MDD architecture, L1-MultiMDD, enriched
with L1-aware speech representation. An end-to-end speech encoder is trained on
the input signal and its corresponding reference phoneme sequence. First, an
attention mechanism is deployed to align the input audio with the reference
phoneme sequence. Afterwards, the L1-L2-speech embedding are extracted from an
auxiliary model, pretrained in a multi-task setup identifying L1 and L2
language, and are infused with the primary network. Finally, the L1-MultiMDD is
then optimized for a unified multilingual phoneme recognition task using
connectionist temporal classification (CTC) loss for the target languages:
English, Arabic, and Mandarin. Our experiments demonstrate the effectiveness of
the proposed L1-MultiMDD framework on both seen -- L2-ARTIC, LATIC, and
AraVoiceL2v2; and unseen -- EpaDB and Speechocean762 datasets. The consistent
gains in PER, and false rejection rate (FRR) across all target languages
confirm our approach's robustness, efficacy, and generalizability.

本文介绍了一种新颖的多语言 MDD 结构 L1-MultiMDD，它具有 L1 感知的语音表示，并使用端到端语音编码器对输入信号及其对应的参考音素序列进行训练。实验结果表明，该 L1-MultiMDD 框架在目标语言（包括英语、阿拉伯语和普通话）的统一多语言音素识别任务中具有鲁棒性、有效性和泛化能力。