Creating Automatic Speech Recognition (ASR) systems that are robust and
resilient to classroom conditions is paramount to the development of AI tools
to aid teachers and students. In this work, we study the efficacy of continued
pretraining (CPT) in adapting Wav2vec2.0 to the classroom domain. We show that
CPT is a powerful tool in that regard and reduces the Word Error Rate (WER) of
Wav2vec2.0-based models by upwards of 10%. More specifically, CPT improves the
model's robustness to different noises, microphones, classroom conditions as
well as classroom demographics. Our CPT models show improved ability to
generalize to different demographics unseen in the labeled finetuning data.

通过持续预训练 (CPT)，我们研究了将 Wav2vec2.0 调整到课堂领域的有效性，结果表明 CPT 是一个强大的工具，可以将基于 Wav2vec2.0 的模型的词错误率（WER）降低 10% 以上，改善了模型对不同噪声、麦克风、课堂条件以及学生群体的适应能力。同时，我们的 CPT 模型在标记微调数据中未见过的不同人群中展示了更好的泛化能力。