Self-Supervised Learning (SSL) models have been successfully applied in
various deep learning-based speech tasks, particularly those with a limited
amount of data. However, the quality of SSL representations depends highly on
the relatedness between the SSL training domain(s) and the target data domain.
On the contrary, spectral feature (SF) extractors such as log Mel-filterbanks
are hand-crafted non-learnable components, and could be more robust to domain
shifts. The present work examines the assumption that combining non-learnable
SF extractors to SSL models is an effective approach to low resource speech
tasks. We propose a learnable and interpretable framework to combine SF and SSL
representations. The proposed framework outperforms significantly both baseline
and SSL models on Automatic Speech Recognition (ASR) and Speech Translation
(ST) tasks on three low resource datasets. We additionally design a mixture of
experts based combination model. This last model reveals that the relative
contribution of SSL models over conventional SF extractors is very small in
case of domain mismatch between SSL training set and the target language data.

该研究旨在探索将非学习的谱特征 (SF) 提取器与自我学习模型相结合的有效性，结果表明，该方法显著优于自我学习模型和基线模型，在语音识别和语音翻译两种任务中都表现出更好的性能，在自我学习培训集和目标语言数据存在域不匹配情况时，自我学习模型的相对贡献非常小。