This paper presents a novel task, zero-shot voice conversion based on face
images (zero-shot FaceVC), which aims at converting the voice characteristics
of an utterance from any source speaker to a newly coming target speaker,
solely relying on a single face image of the target speaker. To address this
task, we propose a face-voice memory-based zero-shot FaceVC method. This method
leverages a memory-based face-voice alignment module, in which slots act as the
bridge to align these two modalities, allowing for the capture of voice
characteristics from face images. A mixed supervision strategy is also
introduced to mitigate the long-standing issue of the inconsistency between
training and inference phases for voice conversion tasks. To obtain
speaker-independent content-related representations, we transfer the knowledge
from a pretrained zero-shot voice conversion model to our zero-shot FaceVC
model. Considering the differences between FaceVC and traditional voice
conversion tasks, systematic subjective and objective metrics are designed to
thoroughly evaluate the homogeneity, diversity and consistency of voice
characteristics controlled by face images. Through extensive experiments, we
demonstrate the superiority of our proposed method on the zero-shot FaceVC
task. Samples are presented on our demo website.

一个基于面部图像的零样本语音转换任务中，提出了一种新颖的零样本面部语音转换方法，通过使用面部 - 语音对齐模块和混合监督策略来实现从一个源说话者到一个目标说话者的语音特征转换，并引入预训练的零样本语音转换模型，通过大量实验证明了该方法在零样本面部语音转换任务中的优越性。

基于面部驱动的零射声音转换与基于记忆的面音对齐

Face-Driven Zero-Shot Voice Conversion with Memory-based Face-Voice  Alignment

In this study, we present a speech corpus of patients with chronic kidney
disease (CKD) that will be used for research on pathological voice analysis,
automatic illness identification, and severity prediction. This paper
introduces the steps involved in creating this corpus, including the choice of
speech-related parameters and speech lists as well as the recording technique.
The speakers in this corpus, 289 CKD patients with varying degrees of severity
who were categorized based on estimated glomerular filtration rate (eGFR),
delivered sustained vowels, sentence, and paragraph stimuli. This study
compared and analyzed the voice characteristics of CKD patients with those of
the control group; the results revealed differences in voice quality,
phoneme-level pronunciation, prosody, glottal source, and aerodynamic
parameters.

本研究介绍了一种用于病理声音分析、自动疾病识别和严重程度预测的慢性肾脏病（CKD）患者语音语料库的创建步骤，比较分析了 289 名慢性肾脏病患者的音色、发音、韵律、声带来源和空气动力学参数等与对照组的声音特征差异。