In recent years, text-to-speech (TTS) technology has witnessed impressive
advancements, particularly with large-scale training datasets, showcasing
human-level speech quality and impressive zero-shot capabilities on unseen
speakers. However, despite human subjective evaluations, such as the mean
opinion score (MOS), remaining the gold standard for assessing the quality of
synthetic speech, even state-of-the-art TTS approaches have kept human feedback
isolated from training that resulted in mismatched training objectives and
evaluation metrics. In this work, we investigate a novel topic of integrating
subjective human evaluation into the TTS training loop. Inspired by the recent
success of reinforcement learning from human feedback, we propose a
comprehensive sampling-annotating-learning framework tailored to TTS
optimization, namely uncertainty-aware optimization (UNO). Specifically, UNO
eliminates the need for a reward model or preference data by directly
maximizing the utility of speech generations while considering the uncertainty
that lies in the inherent variability in subjective human speech perception and
evaluations. Experimental results of both subjective and objective evaluations
demonstrate that UNO considerably improves the zero-shot performance of TTS
models in terms of MOS, word error rate, and speaker similarity. Additionally,
we present a remarkable ability of UNO that it can adapt to the desired
speaking style in emotional TTS seamlessly and flexibly.

在此研究中，我们提出了一种将主观人类评估整合到 TTS 训练过程中的新方法，名为不确定性感知优化（UNO），通过考虑主观人类语音感知和评估中的固有变异性来最大化语音生成的效用，实验证明 UNO 极大地提升了 TTS 模型在 MOS、词错误率和说话人相似性方面的零样本性能，同时还展示了 UNO 在情感 TTS 中无缝、灵活地适应所需说话风格的显著能力。