This paper presents a high quality singing synthesizer that is able to model
a voice with limited available recordings. Based on the sequence-to-sequence
singing model, we design a multi-singer framework to leverage all the existing
singing data of different singers. To attenuate the issue of musical score
unbalance among singers, we incorporate an adversarial task of singer
classification to make encoder output less singer dependent. Furthermore, we
apply multiple random window discriminators (MRWDs) on the generated acoustic
features to make the network be a GAN. Both objective and subjective
evaluations indicate that the proposed synthesizer can generate higher quality
singing voice than baseline (4.12 vs 3.53 in MOS). Especially, the articulation
of high-pitched vowels is significantly enhanced.

本文提出了一种高质量的歌唱合成器，它能够在有限的可用录音基础上模拟出一种声音，采用序列到序列的歌唱模型，并设计了一个多歌手框架来利用不同歌手的现有歌唱数据，以减轻歌唱评分不平衡的问题。此外，为了使编码器输出与歌手无关，我们加入了一个对抗性任务来保证模型的平衡性及多随机窗口鉴别器（MRWDs）以使网络成为 GAN。客观和主观的评估表明，所提出的合成器比基准测试能够产生更高质量的歌唱声音（MOS 值分别为 4.12 和 3.53）。特别是高音元音的表达得到了显著改善。

对抗训练的多歌手序列到序列歌唱合成器

Adversarially Trained Multi-Singer Sequence-To-Sequence Singing  Synthesizer

We propose a sequence-to-sequence singing synthesizer, which avoids the need
for training data with pre-aligned phonetic and acoustic features. Rather than
the more common approach of a content-based attention mechanism combined with
an autoregressive decoder, we use a different mechanism suitable for
feed-forward synthesis. Given that phonetic timings in singing are highly
constrained by the musical score, we derive an approximate initial alignment
with the help of a simple duration model. Then, using a decoder based on a
feed-forward variant of the Transformer model, a series of self-attention and
convolutional layers refines the result of the initial alignment to reach the
target acoustic features. Advantages of this approach include faster inference
and avoiding the exposure bias issues that affect autoregressive models trained
by teacher forcing. We evaluate the effectiveness of this model compared to an
autoregressive baseline, the importance of self-attention, and the importance
of the accuracy of the duration model.

提出了一种基于 Transformer 模型的序列到序列歌曲合成器，使用自身注意力机制结合初始的音长模型来准确合成出作曲家想要的声音，相比于基于老师强制训练的自回归模型，具有更快的推理速度和更好的生成效果。