Previous research has shown that established techniques for spoken voice
conversion (VC) do not perform as well when applied to singing voice conversion
(SVC). We propose an alternative loss component in a loss function that is
otherwise well-established among VC tasks, which has been shown to improve our
model's SVC performance. We first trained a singer identity embedding (SIE)
network on mel-spectrograms of singer recordings to produce singer-specific
variance encodings using contrastive learning. We subsequently trained a
well-known autoencoder framework (AutoVC) conditioned on these SIEs, and
measured differences in SVC performance when using different latent regressor
loss components. We found that using this loss w.r.t. SIEs leads to better
performance than w.r.t. bottleneck embeddings, where converted audio is more
natural and specific towards target singers. The inclusion of this loss
component has the advantage of explicitly forcing the network to reconstruct
with timbral similarity, and also negates the effect of poor disentanglement in
AutoVC's bottleneck embeddings. We demonstrate peculiar diversity between
computational and human evaluations on singer-converted audio clips, which
highlights the necessity of both. We also propose a pitch-matching mechanism
between source and target singers to ensure these evaluations are not
influenced by differences in pitch register.

本文提出一种基于歌唱声音转换（SVC）的损失函数的替代损失组件，该组件通过歌手身份嵌入网络（SIE）来测量音频转换的相似度，在实验中表现出较好的转换效果。

歌声转换的潜在回归损失比较分析

A Comparative Analysis Of Latent Regressor Losses For Singing Voice Conversion

Non-parallel many-to-many voice conversion remains an interesting but
challenging speech processing task. Recently, AutoVC, a conditional autoencoder
based method, achieved excellent conversion results by disentangling the
speaker identity and the speech content using information-constraining
bottlenecks. However, due to the pure autoencoder training method, it is
difficult to evaluate the separation effect of content and speaker identity. In
this paper, a novel voice conversion framework, named $\boldsymbol T$ext
$\boldsymbol G$uided $\boldsymbol A$utoVC(TGAVC), is proposed to more
effectively separate content and timbre from speech, where an expected content
embedding produced based on the text transcriptions is designed to guide the
extraction of voice content. In addition, the adversarial training is applied
to eliminate the speaker identity information in the estimated content
embedding extracted from speech. Under the guidance of the expected content
embedding and the adversarial training, the content encoder is trained to
extract speaker-independent content embedding from speech. Experiments on
AIShell-3 dataset show that the proposed model outperforms AutoVC in terms of
naturalness and similarity of converted speech.

本文提出了一种名为 TGAVC 的语音转换框架，通过望文生义的内容嵌入，辅助提取正文内容并应用对抗训练消除说话人身份信息，从而更有效地分离语音的内容和音色。实验结果表明，TGAVC 模型在自然度和转换语音的相似度上优于 AutoVC。

TGAVC: 使用文本引导和对抗训练来改进自编码器语音转换

TGAVC: Improving Autoencoder Voice Conversion with Text-Guided and Adversarial Training

Any-to-any voice conversion (VC) aims to convert the timbre of utterances
from and to any speakers seen or unseen during training. Various any-to-any VC
approaches have been proposed like AUTOVC, AdaINVC, and FragmentVC. AUTOVC, and
AdaINVC utilize source and target encoders to disentangle the content and
speaker information of the features. FragmentVC utilizes two encoders to encode
source and target information and adopts cross attention to align the source
and target features with similar phonetic content. Moreover, pre-trained
features are adopted. AUTOVC used dvector to extract speaker information, and
self-supervised learning (SSL) features like wav2vec 2.0 is used in FragmentVC
to extract the phonetic content information. Different from previous works, we
proposed S2VC that utilizes Self-Supervised features as both source and target
features for VC model. Supervised phoneme posteriororgram (PPG), which is
believed to be speaker-independent and widely used in VC to extract content
information, is chosen as a strong baseline for SSL features. The objective
evaluation and subjective evaluation both show models taking SSL feature CPC as
both source and target features outperforms that taking PPG as source feature,
suggesting that SSL features have great potential in improving VC.

本文介绍了几种任意语音转换方法（例如 AUTOVC、AdaINVC 和 FragmentVC）以及一种新方法 S2VC，该方法利用自监督学习（SSL）特征作为源和目标特征进行 VC 模型，该方法的客观评价和主观评价都显示出优于以 PPG 为源特征的模型，表明 SSL 特征在改进 VC 方面具有巨大潜力。