This paper describes a baseline for the second iteration of the Fact
Extraction and VERification shared task (FEVER2.0) which explores the
resilience of systems through adversarial evaluation. We present a collection
of simple adversarial attacks against systems that participated in the first
FEVER shared task. FEVER modeled the assessment of truthfulness of written
claims as a joint information retrieval and natural language inference task
using evidence from Wikipedia. A large number of participants made use of deep
neural networks in their submissions to the shared task. The extent as to
whether such models understand language has been the subject of a number of
recent investigations and discussion in literature. In this paper, we present a
simple method of generating entailment-preserving and entailment-altering
perturbations of instances by common patterns within the training data. We find
that a number of systems are greatly affected with absolute losses in
classification accuracy of up to $29\%$ on the newly perturbed instances. Using
these newly generated instances, we construct a sample submission for the
FEVER2.0 shared task. Addressing these types of attacks will aid in building
more robust fact-checking models, as well as suggest directions to expand the
datasets.

本文的 FEVER2.0 基准测试通过生成针对系统的对抗性攻击探索了 NN 模型在事实提取和验证任务中的鲁棒性，结果表明深度神经网络在处理 pattren 异常的样本时效果较差，提出对这类攻击进行研究有助于构建更健壮的事实检查模型，同时建议扩展数据集。