Mispronunciation Detection and Diagnosis (MDD) systems, leveraging Automatic
Speech Recognition (ASR), face two main challenges in Mandarin Chinese: 1) The
two-stage models create an information gap between the phoneme or tone
classification stage and the MDD stage. 2) The scarcity of Mandarin MDD
datasets limits model training. In this paper, we introduce a stateless RNN-T
model for Mandarin MDD, utilizing HuBERT features with pitch embedding through
a Pitch Fusion Block. Our model, trained solely on native speaker data, shows a
3% improvement in Phone Error Rate and a 7% increase in False Acceptance Rate
over the state-of-the-art baseline in non-native scenarios

本研究介绍了一种基于华文的 MDD（Mispronunciation Detection and Diagnosis）系统，采用无状态的 RNN-T 模型，利用带音高嵌入的 HuBERT 特征通过音高融合块进行训练，该模型仅使用母语人士的数据，在非母语场景中，其在电话错误率上表现出 3％的改进和假接受率上的 7％增加，优于现有技术基准。

基于音高感知的 Mandarin Chinese 语音错误检测与诊断的 RNN-T

Pitch-Aware RNN-T for Mandarin Chinese Mispronunciation Detection and  Diagnosis

We revisit a self-supervised method that segments unlabelled speech into
word-like segments. We start from the two-stage duration-penalised dynamic
programming method that performs zero-resource segmentation without learning an
explicit lexicon. In the first acoustic unit discovery stage, we replace
contrastive predictive coding features with HuBERT. After word segmentation in
the second stage, we get an acoustic word embedding for each segment by
averaging HuBERT features. These embeddings are clustered using K-means to get
a lexicon. The result is good full-coverage segmentation with a lexicon that
achieves state-of-the-art performance on the ZeroSpeech benchmarks.

我们重新审视了一种自我监督的方法，将未标记的语音分割成类似词的段落。我们从两阶段的惩罚持续时间的动态规划方法开始，进行零资源分割，而无需学习明确的词汇表。在第一阶段的声学单元发现阶段，我们用 HuBERT 替换对比预测编码特征。在第二阶段的词语分割之后，我们通过平均 HuBERT 特征获得每个段落的声学词嵌入。使用 K-means 对这些嵌入进行聚类，以获得一个词汇表。结果是具有良好覆盖率的分割，其词汇表在 ZeroSpeech 基准测试中达到了最先进的性能。

改进特征的语音分割和词库学习再探

Revisiting speech segmentation and lexicon learning with better features

Recent work in spoken language modeling shows the possibility of learning a
language unsupervisedly from raw audio without any text labels. The approach
relies first on transforming the audio into a sequence of discrete units (or
pseudo-text) and then training a language model directly on such pseudo-text.
Is such a discrete bottleneck necessary, potentially introducing irreversible
errors in the encoding of the speech signal, or could we learn a language model
without discrete units at all? In this work, we study the role of discrete
versus continuous representations in spoken language modeling. We show that
discretization is indeed essential for good results in spoken language
modeling. We show that discretization removes linguistically irrelevant
information from the continuous features, helping to improve language modeling
performances. On the basis of this study, we train a language model on the
discrete units of the HuBERT features, reaching new state-of-the-art results in
the lexical, syntactic and semantic metrics of the Zero Resource Speech
Challenge 2021 (Track 1 - Speech Only).

本文研究了在语音语言建模中，离散与连续表示的作用，结论表明离散化具有重要作用，并在 HuBERT 特征上训练了语言模型，在 Zero Resource Speech Challenge 2021 中实现了最新的语言词汇、句法和语义方面的最佳表现。