Text-to-speech models trained on large-scale datasets have demonstrated
impressive in-context learning capabilities and naturalness. However, control
of speaker identity and style in these models typically requires conditioning
on reference speech recordings, limiting creative applications. Alternatively,
natural language prompting of speaker identity and style has demonstrated
promising results and provides an intuitive method of control. However,
reliance on human-labeled descriptions prevents scaling to large datasets.
Our work bridges the gap between these two approaches. We propose a scalable
method for labeling various aspects of speaker identity, style, and recording
conditions. We then apply this method to a 45k hour dataset, which we use to
train a speech language model. Furthermore, we propose simple methods for
increasing audio fidelity, significantly outperforming recent work despite
relying entirely on found data.
Our results demonstrate high-fidelity speech generation in a diverse range of
accents, prosodic styles, channel conditions, and acoustic conditions, all
accomplished with a single model and intuitive natural language conditioning.
Audio samples can be heard at this https URL

通过对大规模数据集进行训练的文本到语音模型展示了令人印象深刻的语境学习能力和自然度。然而，这些模型中的说话人身份和风格的控制通常需要以参考语音录音为基础，从而限制了其创造性应用。相反，关于说话人身份和风格的自然语言提示已经展示了有希望的结果，并提供了一种直观的控制方法。然而，依赖于人工标注的描述限制了其扩展到大规模数据集的能力。我们的工作弥合了这两种方法之间的差距。我们提出了一种可扩展的方法来对说话人身份、风格和录音条件的各个方面进行标注。然后，我们将这种方法应用到一个 45k 小时的数据集上，用于训练语音语言模型。此外，我们提出了简单的方法来增加音频保真度，尽管完全依赖于现有数据，但性能显著超越了最近的工作。我们的结果展示了通过单一模型和直观的自然语言条件，实现了高保真度的语音生成，在各种口音、韵律风格、信道条件和声学条件下均表现出色。可以在此网址听到音频样本。