The zero-shot text-to-speech (TTS) method, based on speaker embeddings
extracted from reference speech using self-supervised learning (SSL) speech
representations, can reproduce speaker characteristics very accurately.
However, this approach suffers from degradation in speech synthesis quality
when the reference speech contains noise. In this paper, we propose a
noise-robust zero-shot TTS method. We incorporated adapters into the SSL model,
which we fine-tuned with the TTS model using noisy reference speech. In
addition, to further improve performance, we adopted a speech enhancement (SE)
front-end. With these improvements, our proposed SSL-based zero-shot TTS
achieved high-quality speech synthesis with noisy reference speech. Through the
objective and subjective evaluations, we confirmed that the proposed method is
highly robust to noise in reference speech, and effectively works in
combination with SE.

基于自监督学习的说话人嵌入提取的零样本文本到语音方法可以非常准确地再现说话人的特征。然而，当参考语音包含噪声时，这种方法的语音合成质量会降低。本文中，我们提出了一种噪声鲁棒的零样本文本到语音方法。我们将适配器融入自监督学习模型中，并使用带噪声的参考语音对其进行微调，同时采用了语音增强前端以进一步提高性能。通过客观和主观评估，我们证实了所提出方法对参考语音中的噪声具有很高的鲁棒性，并且与语音增强相结合有效地工作。