Self-supervised learning (SSL) speech representations learned from large
amounts of diverse, mixed-quality speech data without transcriptions are
gaining ground in many speech technology applications. Prior work has shown
that SSL is an effective intermediate representation in two-stage
text-to-speech (TTS) for both read and spontaneous speech. However, it is still
not clear which SSL and which layer from each SSL model is most suited for
spontaneous TTS. We address this shortcoming by extending the scope of
comparison for SSL in spontaneous TTS to 6 different SSLs and 3 layers within
each SSL. Furthermore, SSL has also shown potential in predicting the mean
opinion scores (MOS) of synthesized speech, but this has only been done in
read-speech MOS prediction. We extend an SSL-based MOS prediction framework
previously developed for scoring read speech synthesis and evaluate its
performance on synthesized spontaneous speech. All experiments are conducted
twice on two different spontaneous corpora in order to find generalizable
trends. Overall, we present comprehensive experimental results on the use of
SSL in spontaneous TTS and MOS prediction to further quantify and understand
how SSL can be used in spontaneous TTS. Audios samples:
this https URL

本研究探讨了自监督学习在 spontaneous TTS 和 MOS 预测中的适用性，并在实验中比较了 6 种不同的 SSL 及每种模型中的不同层次，旨在深入了解 SSL 的应用。

关于使用自监督语音表示在自发语音合成中的研究

On the Use of Self-Supervised Speech Representations in Spontaneous  Speech Synthesis

MOS (Mean Opinion Score) is a subjective method used for the evaluation of a
system's quality. Telecommunications (for voice and video), and speech
synthesis systems (for generated speech) are a few of the many applications of
the method. While MOS tests are widely accepted, they are time-consuming and
costly since human input is required. In addition, since the systems and
subjects of the tests differ, the results are not really comparable. On the
other hand, a large number of previous tests allow us to train machine learning
models that are capable of predicting MOS value. By automatically predicting
MOS values, both the aforementioned issues can be resolved.
The present work introduces data-, training- and post-training specific
improvements to a previous self-supervised learning-based MOS prediction model.
We used a wav2vec 2.0 model pre-trained on LibriSpeech, extended with LSTM and
non-linear dense layers. We introduced transfer learning, target data
preprocessing a two- and three-phase training method with different batch
formulations, dropout accumulation (for larger batch sizes) and quantization of
the predictions.
The methods are evaluated using the shared synthetic speech dataset of the
first Voice MOS challenge.

该研究通过对先前基于自监督学习的 MOS 预测模型进行数据、训练和后训练的特定改进，并采用多种技术评估其有效性，包括 wav2vec 2.0 模型、转移学习、不同的批处理方法和方法的量化等，实现自动预测 MOS 值。