Recently, zero-shot text-to-speech (TTS) systems, capable of synthesizing any
speaker's voice from a short audio prompt, have made rapid advancements.
However, the quality of the generated speech significantly deteriorates when
the audio prompt contains noise, and limited research has been conducted to
address this issue. In this paper, we explored various strategies to enhance
the quality of audio generated from noisy audio prompts within the context of
flow-matching-based zero-shot TTS. Our investigation includes comprehensive
training strategies: unsupervised pre-training with masked speech denoising,
multi-speaker detection and DNSMOS-based data filtering on the pre-training
data, and fine-tuning with random noise mixing. The results of our experiments
demonstrate significant improvements in intelligibility, speaker similarity,
and overall audio quality compared to the approach of applying speech
enhancement to the audio prompt.

对于从噪声音频提示生成的语音合成，我们研究了多种策略来提高其质量，包括无监督预训练、多说话者检测、基于 DNSMOS 的数据过滤和随机噪声混合微调等，实验证明相较于应用语音增强到音频提示的方法，我们的方法能够显著提高可懂度、说话者相似度以及整体音频质量。