Although voice conversion (VC) systems have shown a remarkable ability to
transfer voice style, existing methods still have an inaccurate pitch and low
speaker adaptation quality. To address these challenges, we introduce
Diff-HierVC, a hierarchical VC system based on two diffusion models. We first
introduce DiffPitch, which can effectively generate F0 with the target voice
style. Subsequently, the generated F0 is fed to DiffVoice to convert the speech
with a target voice style. Furthermore, using the source-filter encoder, we
disentangle the speech and use the converted Mel-spectrogram as a data-driven
prior in DiffVoice to improve the voice style transfer capacity. Finally, by
using the masked prior in diffusion models, our model can improve the speaker
adaptation quality. Experimental results verify the superiority of our model in
pitch generation and voice style transfer performance, and our model also
achieves a CER of 0.83% and EER of 3.29% in zero-shot VC scenarios.

Diff-HierVC 是一种基于两个扩散模型的分层语音转换系统，通过 DiffPitch 有效地生成具有目标音色的 F0，然后使用 DiffVoice 将语音转换为目标音色，通过源 - 滤波器编码器进行信息分离，并在 DiffVoice 中使用转换后的 Mel 频谱图作为数据驱动先验，从而提高语音风格转换能力，通过扩散模型中的遮罩先验，提高说话者适应质量。实验结果验证了 Diff-HierVC 在音高生成和语音风格转换性能上的优越性，在零样本语音转换场景中，其 CER 为 0.83％，EER 为 3.29％。