A song is a combination of singing voice and accompaniment. However, existing
works focus on singing voice synthesis and music generation independently.
Little attention was paid to explore song synthesis. In this work, we propose a
novel task called text-to-song synthesis which incorporating both vocals and
accompaniments generation. We develop Melodist, a two-stage text-to-song method
that consists of singing voice synthesis (SVS) and vocal-to-accompaniment (V2A)
synthesis. Melodist leverages tri-tower contrastive pretraining to learn more
effective text representation for controllable V2A synthesis. A Chinese song
dataset mined from a music website is built up to alleviate data scarcity for
our research. The evaluation results on our dataset demonstrate that Melodist
can synthesize songs with comparable quality and style consistency. Audio
samples can be found in this https URL

本研究提出了一个称为文本到歌曲合成的新任务，结合了歌声和伴奏生成。我们开发了一种名为 Melodist 的两阶段文本到歌曲方法，它包括了歌声合成和声乐到伴奏合成。Melodist 利用三塔对比预训练来学习更有效的文本表示，用于可控的声乐到伴奏合成。我们构建了一个从音乐网站挖掘的中文歌曲数据集，以解决数据稀缺问题。我们数据集的评估结果表明，Melodist 能够合成具有可比质量和风格一致性的歌曲。音频样本可以在此 https 网址中找到。