Previous research has shown that established techniques for spoken voice
conversion (VC) do not perform as well when applied to singing voice conversion
(SVC). We propose an alternative loss component in a loss function that is
otherwise well-established among VC tasks, which has been shown to improve our
model's SVC performance. We first trained a singer identity embedding (SIE)
network on mel-spectrograms of singer recordings to produce singer-specific
variance encodings using contrastive learning. We subsequently trained a
well-known autoencoder framework (AutoVC) conditioned on these SIEs, and
measured differences in SVC performance when using different latent regressor
loss components. We found that using this loss w.r.t. SIEs leads to better
performance than w.r.t. bottleneck embeddings, where converted audio is more
natural and specific towards target singers. The inclusion of this loss
component has the advantage of explicitly forcing the network to reconstruct
with timbral similarity, and also negates the effect of poor disentanglement in
AutoVC's bottleneck embeddings. We demonstrate peculiar diversity between
computational and human evaluations on singer-converted audio clips, which
highlights the necessity of both. We also propose a pitch-matching mechanism
between source and target singers to ensure these evaluations are not
influenced by differences in pitch register.

本文提出一种基于歌唱声音转换（SVC）的损失函数的替代损失组件，该组件通过歌手身份嵌入网络（SIE）来测量音频转换的相似度，在实验中表现出较好的转换效果。