It is shown that many published models for the Stanford Question Answering
Dataset (Rajpurkar et al., 2016) lack robustness, suffering an over 50%
decrease in F1 score during adversarial evaluation based on the AddSent (Jia
and Liang, 2017) algorithm. It has also been shown that retraining models on
data generated by AddSent has limited effect on their robustness. We propose a
novel alternative adversary-generation algorithm, AddSentDiverse, that
significantly increases the variance within the adversarial training data by
providing effective examples that punish the model for making certain
superficial assumptions. Further, in order to improve robustness to AddSent's
semantic perturbations (e.g., antonyms), we jointly improve the model's
semantic-relationship learning capabilities in addition to our
AddSentDiverse-based adversarial training data augmentation. With these
additions, we show that we can make a state-of-the-art model significantly more
robust, achieving a 36.5% increase in F1 score under many different types of
adversarial evaluation while maintaining performance on the regular SQuAD task.

通过提出一个新的对抗生成算法 AddSentDiverse 和改进模型的语义关系学习能力，作者在 Stanford 问题问答数据集上实现了一个近 36.5% 的 F1 得分提升并提高了模型的鲁棒性。