We are interested in a challenging task, Realistic-Music-Score based Singing
Voice Synthesis (RMS-SVS). RMS-SVS aims to generate high-quality singing voices
given realistic music scores with different note types (grace, slur, rest,
etc.). Though significant progress has been achieved, recent singing voice
synthesis (SVS) methods are limited to fine-grained music scores, which require
a complicated data collection pipeline with time-consuming manual annotation to
align music notes with phonemes. Furthermore, these manual annotation destroys
the regularity of note durations in music scores, making fine-grained music
scores inconvenient for composing. To tackle these challenges, we propose
RMSSinger, the first RMS-SVS method, which takes realistic music scores as
input, eliminating most of the tedious manual annotation and avoiding the
aforementioned inconvenience. Note that music scores are based on words rather
than phonemes, in RMSSinger, we introduce word-level modeling to avoid the
time-consuming phoneme duration annotation and the complicated phoneme-level
mel-note alignment. Furthermore, we propose the first diffusion-based pitch
modeling method, which ameliorates the naturalness of existing pitch-modeling
methods. To achieve these, we collect a new dataset containing realistic music
scores and singing voices according to these realistic music scores from
professional singers. Extensive experiments on the dataset demonstrate the
effectiveness of our methods. Audio samples are available at
this https URL

本文提出 RMSSinger 方法，在基于真实音乐分数的歌唱声合成 (RMS-SVS) 领域第一次使用单词级模型，避免了繁琐的音素时长注释和复杂的音素级音符对齐，并进一步提出了第一种基于扩散过程的音高建模方法，在新的数据集上广泛实验，证明了我们方法的有效性。