In this work we present the experiments which lead to the creation of our
BERT and ELECTRA based German language models, GBERT and GELECTRA. By varying
the input training data, model size, and the presence of Whole Word Masking
(WWM) we were able to attain SoTA performance across a set of document
classification and named entity recognition (NER) tasks for both models of base
and large size. We adopt an evaluation driven approach in training these models
and our results indicate that both adding more data and utilizing WWM improve
model performance. By benchmarking against existing German models, we show that
these models are the best German models to date. Our trained models will be
made publicly available to the research community.

本文介绍了我们使用 BERT 和 ELECTRA 基于德语的语言模型 GBERT 和 GELECTRA 的实验，通过改变输入训练数据、模型大小和整个单词掩蔽（WWM）的存在，我们成功实现了在基础和大型模型的文件分类和命名实体识别（NER）任务中取得 SoTA 性能。我们的训练数据采用了 “评估驱动方法”，表明加入更多的数据和使用 WWM 都能提高模型性能。通过对现有德语模型进行基准测试，我们证明这些模型是迄今为止最好的德语模型。我们的训练模型将公开提供给研究社区。