In this paper, we propose an end-to-end Korean singing voice synthesis system
from lyrics and a symbolic melody using the following three novel approaches:
1) phonetic enhancement masking, 2) local conditioning of text and pitch to the
super-resolution network, and 3) conditional adversarial training. The proposed
system consists of two main modules; a mel-synthesis network that generates a
mel-spectrogram from the given input information, and a super-resolution
network that upsamples the generated mel-spectrogram into a linear-spectrogram.
In the mel-synthesis network, phonetic enhancement masking is applied to
generate implicit formant masks solely from the input text, which enables a
more accurate phonetic control of singing voice. In addition, we show that two
other proposed methods -- local conditioning of text and pitch, and conditional
adversarial training -- are crucial for a realistic generation of the human
singing voice in the super-resolution process. Finally, both quantitative and
qualitative evaluations are conducted, confirming the validity of all proposed
methods.

本文提出了一种从歌词和符号旋律生成端到端的韩国歌唱声音合成系统，包括三种新颖方法：1）语音增强掩蔽，2）文本和音高的本地条件对超分辨率网络，3）条件对抗训练。该系统包括两个主要模块：从给定的输入信息生成 mel-spectrogram 的 mel-synthesis 网络和将生成的 mel-spectrogram 上采样成 linear-spectrogram 的超分辨率网络。在 mel-synthesis 网络中，应用语音增强掩蔽仅从输入文本生成隐式谐振峰掩蔽，这使得歌唱声音的语音控制更加准确。此外，本文还表明了本地文本和音高和条件对抗训练是超分辨率过程中生成真实人类歌声的关键。最后，通过定量和定性评估，证实了所有提出方法的有效性。