We propose DiffSpEx, a generative target speaker extraction method based on
score-based generative modelling through stochastic differential equations.
DiffSpEx deploys a continuous-time stochastic diffusion process in the complex
short-time Fourier transform domain, starting from the target speaker source
and converging to a Gaussian distribution centred on the mixture of sources.
For the reverse-time process, a parametrised score function is conditioned on a
target speaker embedding to extract the target speaker from the mixture of
sources. We utilise ECAPA-TDNN target speaker embeddings and condition the
score function alternately on the SDE time embedding and the target speaker
embedding. The potential of DiffSpEx is demonstrated with the WSJ0-2mix
dataset, achieving an SI-SDR of 12.9 dB and a NISQA score of 3.56. Moreover, we
show that fine-tuning a pre-trained DiffSpEx model to a specific speaker
further improves performance, enabling personalisation in target speaker
extraction.

我们提出了 DiffSpEx，一种基于基于分数的生成建模的目标说话人提取方法，通过随机微分方程实现。 DiffSpEx 在复杂的短时傅里叶变换领域中部署连续时间的随机扩散过程，从目标说话人源开始并收敛到以源混合为中心的高斯分布。 对于逆时间过程，一个参数化的分数函数会被调整到一个目标说话人嵌入上，以从多个源的混合中提取目标说话人。我们使用 ECAPA-TDNN 目标说话人嵌入，并交替将分数函数调整到 SDE 时间嵌入和目标说话人嵌入上。我们利用 WSJ0-2mix 数据集展示了 DiffSpEx 的潜力，实现了 12.9 dB 的 SI-SDR 和 3.56 的 NISQA 得分。此外，我们还展示了将预训练的 DiffSpEx 模型微调到特定说话人能进一步提高性能，实现了目标说话人提取中的个性化。