Developing effective spoken language processing systems for low-resource
languages poses several challenges due to the lack of parallel data and limited
resources for fine-tuning models. In this work, we target on improving upon
both text classification and translation of Nigerian Pidgin (Naija) by
collecting a large-scale parallel English-Pidgin corpus and further propose a
framework of cross-lingual adaptive training that includes both continual and
task adaptive training so as to adapt a base pre-trained model to low-resource
languages. Our studies show that English pre-trained language models serve as a
stronger prior than multilingual language models on English-Pidgin tasks with
up to 2.38 BLEU improvements; and demonstrate that augmenting orthographic data
and using task adaptive training with back-translation can have a significant
impact on model performance.

本研究针对低资源语言建设有效的口语处理系统提出了一个跨语言自适应训练框架，包括持续和任务自适应训练，以适应基于预训练模型的自然语言处理任务，采用英语预训练语言模型作为英语 - Pidgin 任务的强先验，取得了最高 2.38 的 BLEU 分数，加入正字法数据增强和反向翻译的任务自适应训练对模型性能有显著影响。