Despite rapid progress in the voice style transfer (VST) field, recent
zero-shot VST systems still lack the ability to transfer the voice style of a
novel speaker. In this paper, we present HierVST, a hierarchical adaptive
end-to-end zero-shot VST model. Without any text transcripts, we only use the
speech dataset to train the model by utilizing hierarchical variational
inference and self-supervised representation. In addition, we adopt a
hierarchical adaptive generator that generates the pitch representation and
waveform audio sequentially. Moreover, we utilize unconditional generation to
improve the speaker-relative acoustic capacity in the acoustic representation.
With a hierarchical adaptive structure, the model can adapt to a novel voice
style and convert speech progressively. The experimental results demonstrate
that our method outperforms other VST models in zero-shot VST scenarios. Audio
samples are available at https://hiervst.github.io/.

本研究提出了一种分层适应性端到端的零样本语音风格转换模型，通过使用分层变分推断和自监督表示在仅使用语音数据集训练模型的情况下，实现了新的发声者的语音风格转换。该模型采用分层适应性生成器进行音调表示和波形音频逐步生成，利用无条件生成来提高声音相对的声学能力。实验结果表明，在零样本语音风格转换场景中，本方法优于其他语音风格转换模型。