This study explores the necessity of performing cross-corpora evaluation for
grammatical error correction (GEC) models. GEC models have been previously
evaluated based on a single commonly applied corpus: the CoNLL-2014 benchmark.
However, the evaluation remains incomplete because the task difficulty varies
depending on the test corpus and conditions such as the proficiency levels of
the writers and essay topics. To overcome this limitation, we evaluate the
performance of several GEC models, including NMT-based (LSTM, CNN, and
transformer) and an SMT-based model, against various learner corpora
(CoNLL-2013, CoNLL-2014, FCE, JFLEG, ICNALE, and KJ). Evaluation results reveal
that the models' rankings considerably vary depending on the corpus, indicating
that single-corpus evaluation is insufficient for GEC models.

本文研究探讨语法纠错（GEC）模型跨语料库评估的必要性。我们评估了多个 GEC 模型（包括 NMT（LSTM、CNN 和 Transformer）和一个 SMT 模型），并发现单一语料库的评估不足以完整评估 GEC 模型的表现。