As deep neural models in NLP become more complex, and as a consequence
opaque, the necessity to interpret them becomes greater. A burgeoning interest
has emerged in rationalizing explanations to provide short and coherent
justifications for predictions. In this position paper, we advocate for a
formal framework for key concepts and properties about rationalizing
explanations to support their evaluation systematically. We also outline one
such formal framework, tailored to rationalizing explanations of increasingly
complex structures, from free-form explanations to deductive explanations, to
argumentative explanations (with the richest structure). Focusing on the
automated fact verification task, we provide illustrations of the use and
usefulness of our formalization for evaluating explanations, tailored to their
varying structures.

深度神经模型在自然语言处理中变得越来越复杂且难以解释，因此解释它们的必要性也越来越大。本文提出了一个正式框架来支持对解释的系统评估，并提供了适用于不同结构的解释的示例，重点是自动事实验证任务。

自动事实验证解释评估框架的探索

Towards a Framework for Evaluating Explanations in Automated Fact  Verification

Neural models for automated fact verification have achieved promising results
thanks to the availability of large, human-annotated datasets. However, for
each new domain that requires fact verification, creating a dataset by manually
writing claims and linking them to their supporting evidence is expensive. We
develop QACG, a framework for training a robust fact verification model by
using automatically generated claims that can be supported, refuted, or
unverifiable from evidence from Wikipedia. QACG generates question-answer pairs
from the evidence and then converts them into different types of claims.
Experiments on the FEVER dataset show that our QACG framework significantly
reduces the demand for human-annotated training data. In a zero-shot scenario,
QACG improves a RoBERTa model's F1 from 50% to 77%, equivalent in performance
to 2K+ manually-curated examples. Our QACG code is publicly available.

使用 Wikipedia 自动产生问题 - 答案对，生成不同类型的声明的 QACG 框架可以训练一个强健的事实验证模型，实验结果表明，在零样本情况下，QACG 能够将 RoBERTa 模型的 F1 从 50％提高到 77％，并且等效于 2000 多个手动筛选的例子。