This study investigates how supervised quality estimation (QE) models of
grammatical error correction (GEC) are affected by the learners' proficiency
with the data. QE models for GEC evaluations in prior work have obtained a high
correlation with manual evaluations. However, when functioning in a real-world
context, the data used for the reported results have limitations because prior
works were biased toward data by learners with relatively high proficiency
levels. To address this issue, we created a QE dataset that includes multiple
proficiency levels and explored the necessity of performing proficiency-wise
evaluation for QE of GEC. Our experiments demonstrated that differences in
evaluation dataset proficiency affect the performance of QE models, and
proficiency-wise evaluation helps create more robust models.

本研究探讨了受监管质量评估模型对语法错误修正的影响，并发现评估数据的熟练程度会影响评估模型的性能，以此确保更加稳健的评估模型。

语言能力对语法错误修正的质量评估的重要性

Proficiency Matters Quality Estimation in Grammatical Error Correction

Grammar error correction (GEC) systems have become ubiquitous in a variety of
software applications, and have started to approach human-level performance for
some datasets. However, very little is known about how to efficiently
personalize these systems to the user's characteristics, such as their
proficiency level and first language, or to emerging domains of text. We
present the first results on adapting a general-purpose neural GEC system to
both the proficiency level and the first language of a writer, using only a few
thousand annotated sentences. Our study is the broadest of its kind, covering
five proficiency levels and twelve different languages, and comparing three
different adaptation scenarios: adapting to the proficiency level only, to the
first language only, or to both aspects simultaneously. We show that tailoring
to both scenarios achieves the largest performance improvement (3.6 F0.5)
relative to a strong baseline.

本研究是在少量带注释的句子基础上，通过将通用神经语法纠错系统适应于作者的熟练程度和母语中，在五种熟练程度和十二种不同语言的比较下，发现同时适应两种条件可以最大程度地提高其性能（3.6 F0.5）。