Grammatical Error Correction (GEC) has been recently modeled using the
sequence-to-sequence framework. However, unlike sequence transduction problems
such as machine translation, GEC suffers from the lack of plentiful parallel
data. We describe two approaches for generating large parallel datasets for GEC
using publicly available Wikipedia data. The first method extracts
source-target pairs from Wikipedia edit histories with minimal filtration
heuristics, while the second method introduces noise into Wikipedia sentences
via round-trip translation through bridge languages. Both strategies yield
similar sized parallel corpora containing around 4B tokens. We employ an
iterative decoding strategy that is tailored to the loosely supervised nature
of our constructed corpora. We demonstrate that neural GEC models trained using
either type of corpora give similar performance. Fine-tuning these models on
the Lang-8 corpus and ensembling allows us to surpass the state of the art on
both the CoNLL-2014 benchmark and the JFLEG task. We provide systematic
analysis that compares the two approaches to data generation and highlights the
effectiveness of ensembling.

本研究使用了序列到序列的框架对语法纠错进行建模。通过两种方法生成了大量的 Wikipedia 平行数据集，并使用迭代解码策略训练了神经语法纠错模型，最终在 CoNLL-2014 基准和 JFLEG 任务中实现了领先于现有技术的表现。