Progress in neural grammatical error correction (GEC) is hindered by the lack
of annotated training data. Sufficient amounts of high-quality manually
annotated data are not available, so recent research has relied on generating
synthetic data, pretraining on it, and then fine-tuning on real datasets;
performance gains have been achieved either by ensembling or by using huge
pretrained models such as XXL-T5 as the backbone. In this work, we explore an
orthogonal direction: how to use available data more efficiently. First, we
propose auxiliary tasks that exploit the alignment between the original and
corrected sentences, such as predicting a sequence of corrections. We formulate
each task as a sequence-to-sequence problem and perform multi-task training.
Second, we discover that the order of datasets used for training and even
individual instances within a dataset may have important effects on the final
performance, so we set out to find the best training schedule. Together, these
two ideas lead to significant improvements, producing results that improve
state of the art with much smaller models; in particular, we outperform the
best models based on T5-XXL (11B parameters) with a BART-based model (400M
parameters).

本研究通过使用辅助任务和优化训练顺序，针对神经语法错误校正的问题，提出了一种高效利用数据的方法，利用较小规模的基于 BART 模型（400M 参数）的结果优于基于 T5-XXL 模型（11B 参数）的最佳模型。

通过多任务训练和优化训练计划实现高效的语法错误校正

Efficient Grammatical Error Correction Via Multi-Task Training and  Optimized Training Schedule

Synthetic data generation is widely known to boost the accuracy of neural
grammatical error correction (GEC) systems, but existing methods often lack
diversity or are too simplistic to generate the broad range of grammatical
errors made by human writers. In this work, we use error type tags from
automatic annotation tools such as ERRANT to guide synthetic data generation.
We compare several models that can produce an ungrammatical sentence given a
clean sentence and an error type tag. We use these models to build a new, large
synthetic pre-training data set with error tag frequency distributions matching
a given development set. Our synthetic data set yields large and consistent
gains, improving the state-of-the-art on the BEA-19 and CoNLL-14 test sets. We
also show that our approach is particularly effective in adapting a GEC system,
trained on mixed native and non-native English, to a native English test set,
even surpassing real training data consisting of high-quality sentence pairs.

本研究使用自动注释工具 ERRANT 的错误类型标签指导合成数据生成，并比较了多种模型以研究如何基于错误类型标签从正确的句子生成不合语法的句子。在此基础上，构建一个规模较大的合成数据集，其错误标签的频率分布与给定开发集相匹配。该合成数据集具有较好的性能表现，可以显著提高 BEA-19 和 CoNLL14 测试集的性能，并且可以有效地适应 GEC 系统，特别是对于混合本地和非本地语言的英语人群，其效果优于由高质量句对组成的真实训练数据。