We investigate the problem of explainability in machine learning.To address
this problem, Feature Attribution Methods (FAMs) measure the contribution of
each feature through a perturbation test, where the difference in prediction is
compared under different perturbations.However, such perturbation tests may not
accurately distinguish the contributions of different features, when their
change in prediction is the same after perturbation.In order to enhance the
ability of FAMs to distinguish different features' contributions in this
challenging setting, we propose to utilize the probability (PNS) that
perturbing a feature is a necessary and sufficient cause for the prediction to
change as a measure of feature importance.Our approach, Feature Attribution
with Necessity and Sufficiency (FANS), computes the PNS via a perturbation test
involving two stages (factual and interventional).In practice, to generate
counterfactual samples, we use a resampling-based approach on the observed
samples to approximate the required conditional distribution.Finally, we
combine FANS and gradient-based optimization to extract the subset with the
largest PNS.We demonstrate that FANS outperforms existing feature attribution
methods on six benchmarks.

机器学习中解释性问题的研究，通过特征归因方法（FAMs）测量每个特征的贡献，提出利用概率作为特征重要性的度量，通过 Necessity and Sufficiency（FANS）方法进行特征归因，最终与梯度优化相结合提取重要子集，在六个基准测试中 FANS 优于现有特征归因方法。

通过双阶段干扰测试的必要性和充分性特征归因方法对因果解释

Feature Attribution with Necessity and Sufficiency via Dual-stage  Perturbation Test for Causal Explanation

Attention-based transformer models have achieved remarkable progress in
multi-modal tasks, such as visual question answering. The explainability of
attention-based methods has recently attracted wide interest as it can explain
the inner changes of attention tokens by accumulating relevancy across
attention layers. Current methods simply update relevancy by equally
accumulating the token relevancy before and after the attention processes.
However, the importance of token values is usually different during relevance
accumulation. In this paper, we propose a weighted relevancy strategy, which
takes the importance of token values into consideration, to reduce distortion
when equally accumulating relevance. To evaluate our method, we propose a
unified CLIP-based two-stage model, named CLIPmapper, to process
Vision-and-Language tasks through CLIP encoder and a following mapper.
CLIPmapper consists of self-attention, cross-attention, single-modality, and
cross-modality attention, thus it is more suitable for evaluating our generic
explainability method. Extensive perturbation tests on visual question
answering and image captioning validate that our explainability method
outperforms existing methods.

我们提出了一种加权相关性策略，考虑了令牌值的重要性，以减少在平等累积相关性时的失真。通过 CLIP 编码器和后续的映射器，我们提出了统一的基于 CLIP 的两阶段模型，命名为 CLIPmapper，用于处理视觉和语言任务，通过自注意、交叉注意、单模态和交叉模态注意，这使得我们的通用可解释性方法更加合适。对视觉问答和图像字幕进行的广泛扰动测试证实了我们的可解释性方法优于现有的方法。