The performance of acoustic models degrades notably in noisy environments. Speech enhancement (SE) can be used as a front-end strategy to aid automatic speech recognition (ASR) systems. However, existing training objectives of SE methods are not fully effective at integrating speech-text and noisy-clean paired data for training toward unseen ASR systems. In this study, we propose a general denoising framework, D4AM, for various downstream acoustic models. Our framework fine-tunes the SE model with the backward gradient according to a specific acoustic model and the corresponding classification objective. In addition, our method aims to consider the regression objective as an auxiliary loss to make the SE model generalize to other unseen acoustic models. To jointly train an SE unit with regression and classification objectives, D4AM uses an adjustment scheme to directly estimate suitable weighting coefficients rather than undergoing a grid search process with additional training costs. The adjustment scheme consists of two parts: gradient calibration and regression objective weighting. The experimental results show that D4AM can consistently and effectively provide improvements to various unseen acoustic models and outperforms other combination setups. Specifically, when evaluated on the Google ASR API with real noisy data completely unseen during SE training, D4AM achieves a relative WER reduction of 24.65% compared with the direct feeding of noisy input. To our knowledge, this is the first work that deploys an effective combination scheme of regression (denoising) and classification (ASR) objectives to derive a general pre-processor applicable to various unseen ASR systems. Our code is available at https://github.com/ChangLee0903/D4AM.

在嘈杂环境中，声学模型的性能明显下降。本研究提出了一种用于各种下游声学模型的通用降噪框架D4AM，通过特定声学模型和相应的分类目标，用反向梯度调整语音增强模型并考虑回归目标作为辅助损失，该方法可以直接估计合适的权重系数，从而避免了额外的训练成本，并有效改善各种未知声学模型的性能。该框架在Google ASR API上进行实验，相对于直接使用嘈杂输入，取得了24.65%的相对WER降低，是首个实现回归和分类目标有效组合的通用预处理器适用于各种未知ASR系统的工作。

D4AM：用于下游声学模型的通用去噪框架