Recent progress in the task of Grammatical Error Correction (GEC) has been
driven by addressing data sparsity, both through new methods for generating
large and noisy pretraining data and through the publication of small and
higher-quality finetuning data in the BEA-2019 shared task. Building upon
recent work in Neural Machine Translation (NMT), we make use of both kinds of
data by deriving example-level scores on our large pretraining data based on a
smaller, higher-quality dataset. In this work, we perform an empirical study to
discover how to best incorporate delta-log-perplexity, a type of example
scoring, into a training schedule for GEC. In doing so, we perform experiments
that shed light on the function and applicability of delta-log-perplexity.
Models trained on scored data achieve state-of-the-art results on common GEC
test sets.

本文通过使用新方法生成大量嘈杂的预训练数据以及发布质量更高的有限调整数据，解决了语法错误纠正领域的数据稀缺问题；同时，通过在小型、高质量数据集上导出大型预训练数据的示例级分数，针对 delta-log-perplexity 进行实证研究，以最佳方式将其纳入 GEC 的训练计划中，从而实现对 GEC 测试集的最佳结果。