Nowadays, data augmentation through synthetic data has been widely used in
the field of Grammatical Error Correction (GEC) to alleviate the problem of
data scarcity. However, these synthetic data are mainly used in the
pre-training phase rather than the data-limited fine-tuning phase due to
inconsistent error distribution and noisy labels. In this paper, we propose a
synthetic data construction method based on contextual augmentation, which can
ensure an efficient augmentation of the original data with a more consistent
error distribution. Specifically, we combine rule-based substitution with
model-based generation, using the generative model to generate a richer context
for the extracted error patterns. Besides, we also propose a relabeling-based
data cleaning method to mitigate the effects of noisy labels in synthetic data.
Experiments on CoNLL14 and BEA19-Test show that our proposed augmentation
method consistently and substantially outperforms strong baselines and achieves
the state-of-the-art level with only a few synthetic data.

提出一种基于上下文增强的合成数据构建方法，用于语法错误校正，在生成模型的基础上结合基于规则的替换，以增加原始数据的错误分布一致性，并提出一种基于重标记的数据清洗方法来减小合成数据中噪声标签的影响。实验证明，该增强方法在有限的合成数据下始终优于强基线方法，并达到了最先进水平。

通过上下文数据增强来改进语法错误修正

Improving Grammatical Error Correction via Contextual Data Augmentation

We propose a novel data augmentation for labeled sentences called contextual
augmentation. We assume an invariance that sentences are natural even if the
words in the sentences are replaced with other words with paradigmatic
relations. We stochastically replace words with other words that are predicted
by a bi-directional language model at the word positions. Words predicted
according to a context are numerous but appropriate for the augmentation of the
original words. Furthermore, we retrofit a language model with a
label-conditional architecture, which allows the model to augment sentences
without breaking the label-compatibility. Through the experiments for six
various different text classification tasks, we demonstrate that the proposed
method improves classifiers based on the convolutional or recurrent neural
networks.

本文介绍了一种新的数据增强技术 —— 上下文增强，使用双向语言模型预测单词位置上各个单词，将其替换为预测出的单词增强原句子，同时使用带标签的模型改进以确保模型增强句子不破坏标签状态，并在六种不同的文本分类任务中实验证明该方法可以改善基于卷积或循环神经网络的分类器。