Grammatical error correction (GEC) is the task of correcting typos, spelling,
punctuation and grammatical issues in text. Approaching the problem as a
sequence-to-sequence task, we compare the use of a common subword unit
vocabulary and byte-level encoding. Initial synthetic training data is created
using an error-generating pipeline, and used for finetuning two subword-level
models and one byte-level model. Models are then finetuned further on
hand-corrected error corpora, including texts written by children, university
students, dyslexic and second-language writers, and evaluated over different
error types and origins. We show that a byte-level model enables higher
correction quality than a subword approach, not only for simple spelling
errors, but also for more complex semantic, stylistic and grammatical issues.
In particular, initial training on synthetic corpora followed by finetuning on
a relatively small parallel corpus of real-world errors helps the byte-level
model correct a wide range of commonly occurring errors. Our experiments are
run for the Icelandic language but should hold for other similar languages,
particularly morphologically rich ones.

本文讨论了使用不同子词单元和字节级编码方法对语法错误校正任务进行序列到序列学习的影响。使用含错误生成管道的合成数据和手动更正的真实数据对不同模型进行训练和微调，证明了字节级编码方法不仅可以更好地纠正简单的拼写错误，还可以在语义、风格和语法等方面更好地进行纠错，尤其适用于富有形态学特征的语言。