Recent studies have revealed that grammatical error correction methods in the
sequence-to-sequence paradigm are vulnerable to adversarial attack, and simply
utilizing adversarial examples in the pre-training or post-training process can
significantly enhance the robustness of GEC models to certain types of attack
without suffering too much performance loss on clean data. In this paper, we
further conduct a thorough robustness evaluation of cutting-edge GEC methods
for four different types of adversarial attacks and propose a simple yet very
effective Cycle Self-Augmenting (CSA) method accordingly. By leveraging the
augmenting data from the GEC models themselves in the post-training process and
introducing regularization data for cycle training, our proposed method can
effectively improve the model robustness of well-trained GEC models with only a
few more training epochs as an extra cost. More concretely, further training on
the regularization data can prevent the GEC models from over-fitting on
easy-to-learn samples and thus can improve the generalization capability and
robustness towards unseen data (adversarial noise/samples). Meanwhile, the
self-augmented data can provide more high-quality pseudo pairs to improve model
performance on the original testing data. Experiments on four benchmark
datasets and seven strong models indicate that our proposed training method can
significantly enhance the robustness of four types of attacks without using
purposely built adversarial examples in training. Evaluation results on clean
data further confirm that our proposed CSA method significantly improves the
performance of four baselines and yields nearly comparable results with other
state-of-the-art models. Our code is available at
this https URL

本文研究了语法纠错方法在序列到序列范式中容易受到对抗攻击的问题，并提出了一种简单而非常有效的 Cycle Self-Augmenting (CSA) 方法来增强模型的鲁棒性，该方法通过在后期训练过程中利用语法纠错模型自身的数据以及引入循环训练的正则化数据，可以在只增加少量额外训练轮次的情况下有效提高模型的鲁棒性。实验结果表明，该方法可以明显增强四种类型攻击的鲁棒性，并且在干净数据上的评估结果进一步证明了该方法对于改进性能的有效性。