Recent work on self-supervised pre-training focus on leveraging large-scale
unlabeled speech data to build robust end-to-end (E2E) acoustic models (AM)
that can be later fine-tuned on downstream tasks e.g., automatic speech
recognition (ASR). Yet, few works investigated the impact on performance when
the data properties substantially differ between the pre-training and
fine-tuning phases, termed domain shift. We target this scenario by analyzing
the robustness of Wav2Vec 2.0 and XLS-R models on downstream ASR for a
completely unseen domain, air traffic control (ATC) communications. We
benchmark these two models on several open-source and challenging ATC databases
with signal-to-noise ratio between 5 and 20 dB. Relative word error rate (WER)
reductions between 20% to 40% are obtained in comparison to hybrid-based ASR
baselines by only fine-tuning E2E acoustic models with a smaller fraction of
labeled data. We analyze WERs on the low-resource scenario and gender bias
carried by one ATC dataset.

本研究旨在探讨在预训练和 fine-tuning 阶段数据属性存在实质性差异的情况下，在空中交通管制数据上分析 Wav2Vec 2.0 和 XLS-R 模型的鲁棒性，结果显示，通过只使用已标注数据的较小部分来对 E2E 声学模型进行 fine-tuning，相对于基于混合式 ASR 基线的声学模型，可以在几个开源且具有挑战性的 ATC 数据库中实现 20％至 40％的相对字错率降低，并分析了一种 ATC 数据集中的低资源和性别偏见的情况。