Grammatical error correction can be viewed as a low-resource
sequence-to-sequence task, because publicly available parallel corpora are
limited. To tackle this challenge, we first generate erroneous versions of
large unannotated corpora using a realistic noising function. The resulting
parallel corpora are subsequently used to pre-train Transformer models. Then,
by sequentially applying transfer learning, we adapt these models to the domain
and style of the test set. Combined with a context-aware neural spellchecker,
our system achieves competitive results in both restricted and low resource
tracks in ACL 2019 BEA Shared Task. We release all of our code and materials
for reproducibility.

本文介绍了一种用于纠正语法错误的低资源序列到序列任务方法，首先使用现实的噪声函数生成大型未注释语料库的错误版本，然后利用这些并行语料库进行 Transformer 模型的预训练，并通过顺序迁移学习将这些模型适应于测试集的领域和风格。同时结合上下文感知的神经拼写检查器，我们的系统在 ACL 2019 BEA 共享任务中取得了竞争力的结果。我们公开代码与材料以便复现。