Voice conversion aims to convert source speech into a target voice using
recordings of the target speaker as a reference. Newer models are producing
increasingly realistic output. But what happens when models are fed with
non-standard data, such as speech from a user with a speech impairment? We
investigate how a recent voice conversion model performs on non-standard
downstream voice conversion tasks. We use a simple but robust approach called
k-nearest neighbors voice conversion (kNN-VC). We look at four non-standard
applications: stuttered voice conversion, cross-lingual voice conversion,
musical instrument conversion, and text-to-voice conversion. The latter
involves converting to a target voice specified through a text description,
e.g. "a young man with a high-pitched voice". Compared to an established
baseline, we find that kNN-VC retains high performance in stuttered and
cross-lingual voice conversion. Results are more mixed for the musical
instrument and text-to-voice conversion tasks. E.g., kNN-VC works well on some
instruments like drums but not on others. Nevertheless, this shows that voice
conversion models - and kNN-VC in particular - are increasingly applicable in a
range of non-standard downstream tasks. But there are still limitations when
samples are very far from the training distribution. Code, samples, trained
models: this https URL

使用 k 最近邻声音转换模型，调查其在非标准下游语音转换任务中的表现，包括口吃声转换、跨语言声音转换、乐器转换和文本转语音转换。与基准线方法相比，k 最近邻声音转换在口吃声转换和跨语言声音转换中保持高性能，但在乐器转换和文本转语音转换任务中表现有所不同。然而，这表明声音转换模型，尤其是 k 最近邻声音转换，在各种非标准下游任务中逐渐具有应用价值，但在与训练分布相差较大的样本上仍存在一些限制。