With recent advances of diffusion model, generative speech enhancement (SE)
has attracted a surge of research interest due to its great potential for
unseen testing noises. However, existing efforts mainly focus on inherent
properties of clean speech for inference, underexploiting the varying noise
information in real-world conditions. In this paper, we propose a noise-aware
speech enhancement (NASE) approach that extracts noise-specific information to
guide the reverse process in diffusion model. Specifically, we design a noise
classification (NC) model to produce acoustic embedding as a noise conditioner
for guiding the reverse denoising process. Meanwhile, a multi-task learning
scheme is devised to jointly optimize SE and NC tasks, in order to enhance the
noise specificity of extracted noise conditioner. Our proposed NASE is shown to
be a plug-and-play module that can be generalized to any diffusion SE models.
Experiment evidence on VoiceBank-DEMAND dataset shows that NASE achieves
significant improvement over multiple mainstream diffusion SE models,
especially on unseen testing noises.

本文提出了一种噪声感知语音增强方法，通过噪声分类模型提取噪声特定信息来指导扩散模型中的逆向去噪过程，并采用多任务学习模式来优化增强和分类任务，实验表明该方法在 VoiceBank-DEAMND 数据集上的性能显著优于多种扩散型语音增强方法，尤其是在未知噪声上表现出较好的泛化性能。