Creating neural text encoders for written Swiss German is challenging due to a dearth of training data combined with dialectal variation. In this paper, we build on several existing multilingual encoders and adapt them to Swiss German using continued pre-training. Evaluation on three diverse downstream tasks shows that simply adding a Swiss German adapter to a modular encoder achieves 97.5% of fully monolithic adaptation performance. We further find that for the task of retrieving Swiss German sentences given Standard German queries, adapting a character-level model is more effective than the other adaptation strategies. We release our code and the models trained for our experiments at https://github.com/ZurichNLP/swiss-german-text-encoders

在这篇研究中，我们使用继续预训练的方法利用现有的多语言编码器，并添加瑞士德语适配器来适应瑞士德语，从而完成了面临的挑战。通过在三个不同的下游任务上进行评估，我们发现仅仅将瑞士德语适配器添加到模块化编码器中就可以实现完全单体化适配性能的97.5％。此外，对于给定标准德语查询的情况下检索瑞士德语句子的任务，我们发现调整字符级别模型比其他适应策略更为有效。我们将我们的代码和进行实验的模型公开发布于此 https URL。

适应瑞士德语方言的多语言编码器的模块化调整