We are interested in a novel task, singing voice beautifying (SVB). Given the
singing voice of an amateur singer, SVB aims to improve the intonation and
vocal tone of the voice, while keeping the content and vocal timbre. Current
automatic pitch correction techniques are immature, and most of them are
restricted to intonation but ignore the overall aesthetic quality. Hence, we
introduce Neural Singing Voice Beautifier (NSVB), the first generative model to
solve the SVB task, which adopts a conditional variational autoencoder as the
backbone and learns the latent representations of vocal tone. In NSVB, we
propose a novel time-warping approach for pitch correction: Shape-Aware Dynamic
Time Warping (SADTW), which ameliorates the robustness of existing time-warping
approaches, to synchronize the amateur recording with the template pitch curve.
Furthermore, we propose a latent-mapping algorithm in the latent space to
convert the amateur vocal tone to the professional one. To achieve this, we
also propose a new dataset containing parallel singing recordings of both
amateur and professional versions. Extensive experiments on both Chinese and
English songs demonstrate the effectiveness of our methods in terms of both
objective and subjective metrics. Audio samples are available
at~\url{this https URL}. Codes:
\url{this https URL}.

本研究通过引入条件变分自动编码器作为神经网络，提出了一种名为 NSVB 的算法以解决唱歌者专业音色的问题，主要包括时间伸缩技术和基于隐变量的映射算法，该算法能够有效地进行音高校正并且在客观和主观尺度上均获得了更好的实验效果。

学习歌曲之美：神经唱声美化器

Learning the Beauty in Songs: Neural Singing Voice Beautifier

We introduce a data-driven approach to automatic pitch correction of solo
singing performances. The proposed approach predicts note-wise pitch shifts
from the relationship between the respective spectrograms of the singing and
accompaniment. This approach differs from commercial systems, where vocal track
notes are usually shifted to be centered around pitches in a user-defined
score, or mapped to the closest pitch among the twelve equal-tempered scale
degrees. The proposed system treats pitch as a continuous value rather than
relying on a set of discretized notes found in musical scores, thus allowing
for improvisation and harmonization in the singing performance. We train our
neural network model using a dataset of 4,702 amateur karaoke performances
selected for good intonation. Our model is trained on both incorrect
intonation, for which it learns a correction, and intentional pitch variation,
which it learns to preserve. The proposed deep neural network with gated
recurrent units on top of convolutional layers shows promising performance on
the real-world score-free singing pitch correction task of autotuning.

本论文介绍了一种基于数据驱动的自动音高校正方法，该方法利用歌唱声音和伴奏音频相应的频谱图之间的关系来预测音符级别的音高变化。该方法通过神经网络模型进行训练，包括卷积层和门控循环单元，可适用于自由即兴和协调音乐演唱。

Deep Autotuner: 用于歌唱表演的音高校准网络

Deep Autotuner: a Pitch Correcting Network for Singing Performances

Expressive singing voice correction is an appealing but challenging problem.
A robust time-warping algorithm which synchronizes two singing recordings can
provide a promising solution. We thereby propose to address the problem by
canonical time warping (CTW) which aligns amateur singing recordings to
professional ones. A new pitch contour is generated given the alignment
information, and a pitch-corrected singing is synthesized back through the
vocoder. The objective evaluation shows that CTW is robust against
pitch-shifting and time-stretching effects, and the subjective test
demonstrates that CTW prevails the other methods including DTW and the
commercial auto-tuning software. Finally, we demonstrate the applicability of
the proposed method in a practical, real-world scenario.

本研究提出了一种基于 CTW 的唱歌时间对齐算法，将业余表演者的唱歌录音与职业唱歌者的进行对齐以提高唱歌表现，结果显示 CTW 相比其他方法更为鲁棒、更能解决音高失真等问题。