A common approach to explain NLP models, is to use importance measures that
express which tokens are important for a prediction. Unfortunately, such
explanations are often wrong despite being persuasive. Therefore, it is
essential to measure their faithfulness. One such metric is if tokens are truly
important, then masking them should result in worse model performance. However,
token masking introduces out-of-distribution issues and existing solutions are
computationally expensive and employ proxy-models. Furthermore, other metrics
are very limited in scope. In this work, we propose an inherently faithfulness
measurable model that addresses these challenges. This is achieved by using a
novel fine-tuning method that incorporates masking, such that masking tokens
become in-distribution by design. This differs from existing approaches, which
are completely model-agnostic but are inapplicable in practice. We demonstrate
the generality of our approach by applying it to various tasks and validate it
using statistical in-distribution tests. Additionally, because masking is
in-distribution, importance measures which themselves use masking become more
faithful, thus our model becomes more explainable.

提议基于掩码方法的一种内在的可靠度可衡量模型，该模型能够解决现有方法受限、计算昂贵且基于代理模型的问题，通过在设计上使掩码令牌成为数据分布内的方式，从而能够更有效地解释和说明 NLP 模型。

可衡量忠诚度的遮蔽语言模型

Faithfulness Measurable Masked Language Models

The ability to cope accurately and fast with Out-Of-Distribution (OOD)
samples is crucial in real-world safety demanding applications. In this work we
first study the interplay between critical points of 3D point clouds and OOD
samples. Our findings are that common corruptions and outliers are often
interpreted as critical points. We generalize the notion of critical points
into importance measures. We show that training a classification network based
only on less important points dramatically improves robustness, at a cost of
minor performance loss on the clean set. We observe that normalized entropy is
highly informative for corruption analysis. An adaptive threshold based on
normalized entropy is suggested for selecting the set of uncritical points. Our
proposed importance measure is extremely fast to compute. We show it can be
used for a variety of applications, such as Explainable AI (XAI), Outlier
Removal, Uncertainty Estimation, Robust Classification and Adversarial Defense.
We reach SOTA results on the two latter tasks.

在这项研究中，我们研究了三维点云的关键点与样本外分布的相互作用，将关键点的概念扩展为重要性度量，通过仅基于较不重要的点进行训练，大大提高了网络的鲁棒性，同时对于协同学习是高度有价值的。

关键点 ++：基于敏捷性点云重要性度量的鲁棒分类、对抗性防御和可解释人工智能

Critical Points ++: An Agile Point Cloud Importance Measure for Robust  Classification, Adversarial Defense and Explainable AI

To explain NLP models a popular approach is to use importance measures, such
as attention, which inform input tokens are important for making a prediction.
However, an open question is how well these explanations accurately reflect a
model's logic, a property called faithfulness.
To answer this question, we propose Recursive ROAR, a new faithfulness
metric. This works by recursively masking allegedly important tokens and then
retraining the model. The principle is that this should result in worse model
performance compared to masking random tokens. The result is a performance
curve given a masking-ratio. Furthermore, we propose a summarizing metric using
relative area-between-curves (RACU), which allows for easy comparison across
papers, models, and tasks.
We evaluate 4 different importance measures on 8 different datasets, using
both LSTM-attention models and RoBERTa models. We find that the faithfulness of
importance measures is both model-dependent and task-dependent. This conclusion
contradicts previous evaluations in both computer vision and faithfulness of
attention literature.

本文提出了一种新的忠实度度量标准，即递归 ROAR，并对四种不同的重要性评估方法在八个数据集上进行了评估，发现其忠实度既与模型相关，也与任务相关。

通过递归屏蔽所谓重要的令牌并重新训练来评估 NLP 中重要性度量的忠实度

Evaluating the Faithfulness of Importance Measures in NLP by Recursively  Masking Allegedly Important Tokens and Retraining

Recent network pruning methods focus on pruning models early-on in training.
To estimate the impact of removing a parameter, these methods use importance
measures that were originally designed to prune trained models. Despite lacking
justification for their use early-on in training, such measures result in
surprisingly low accuracy loss. To better explain this behavior, we develop a
general framework that uses gradient flow to unify state-of-the-art importance
measures through the norm of model parameters. We use this framework to
determine the relationship between pruning measures and evolution of model
parameters, establishing several results related to pruning models early-on in
training: (i) magnitude-based pruning removes parameters that contribute least
to reduction in loss, resulting in models that converge faster than
magnitude-agnostic methods; (ii) loss-preservation based pruning preserves
first-order model evolution dynamics and is therefore appropriate for pruning
minimally trained models; and (iii) gradient-norm based pruning affects
second-order model evolution dynamics, such that increasing gradient norm via
pruning can produce poorly performing models. We validate our claims on several
VGG-13, MobileNet-V1, and ResNet-56 models trained on CIFAR-10/CIFAR-100. Code
available at this https URL

该篇研究通过梯度流的方式，研究了深度神经网络早期剪枝的影响，通过多组实验验证了几种不同的影响度量标准，从而在神经网络剪枝的早期阶段提出了一种更加有效的剪枝方法。