We present a neural analysis and synthesis (NANSY) framework that can
manipulate voice, pitch, and speed of an arbitrary speech signal. Most of the
previous works have focused on using information bottleneck to disentangle
analysis features for controllable synthesis, which usually results in poor
reconstruction quality. We address this issue by proposing a novel training
strategy based on information perturbation. The idea is to perturb information
in the original input signal (e.g., formant, pitch, and frequency response),
thereby letting synthesis networks selectively take essential attributes to
reconstruct the input signal. Because NANSY does not need any bottleneck
structures, it enjoys both high reconstruction quality and controllability.
Furthermore, NANSY does not require any labels associated with speech data such
as text and speaker information, but rather uses a new set of analysis
features, i.e., wav2vec feature and newly proposed pitch feature, Yingram,
which allows for fully self-supervised training. Taking advantage of fully
self-supervised training, NANSY can be easily extended to a multilingual
setting by simply training it with a multilingual dataset. The experiments show
that NANSY can achieve significant improvement in performance in several
applications such as zero-shot voice conversion, pitch shift, and time-scale
modification.

提出了一种神经分析合成（NANSY）框架，它可以操作任意语音信号的声音、音高和速度，通过使用信息扰动提出了一种新的训练策略，使 NANSY 具有高重构质量和可控性，不需要任何瓶颈结构，使用全自监督训练方式，可以在多个应用领域获得显著的性能提升。