Neural networks (NNs) can learn to rely on spurious signals in the training
data, leading to poor generalisation. Recent methods tackle this problem by
training NNs with additional ground-truth annotations of such signals. These
methods may, however, let spurious signals re-emerge in deep convolutional NNs
(CNNs). We propose Targeted Activation Penalty (TAP), a new method tackling the
same problem by penalising activations to control the re-emergence of spurious
signals in deep CNNs, while also lowering training times and memory usage. In
addition, ground-truth annotations can be expensive to obtain. We show that TAP
still works well with annotations generated by pre-trained models as effective
substitutes of ground-truth annotations. We demonstrate the power of TAP
against two state-of-the-art baselines on the MNIST benchmark and on two
clinical image datasets, using four different CNN architectures.

神经网络（NN）可以学习依赖于训练数据中的虚假信号，从而导致很差的泛化能力。最近的研究针对该问题使用附加的虚假信号的真实注释来训练 NNs。然而，这些方法可能导致虚假信号在深度卷积 NNs（CNNs）中重新出现。我们提出了一种新方法，即目标激活惩罚（TAP），通过罚则来控制深度 CNNs 中虚假信号的重新出现，同时降低训练时间和内存使用。此外，真实注释可能需要费用较高。我们展示了 TAP 对 MNIST 基准测试和两个临床图像数据集的两种最先进方法的优势，使用四种不同的 CNN 架构，并表明 TAP 仍然适用于由预训练模型生成的注释作为真实注释的有效替代。

有针对性的激活罚则帮助 CNN 忽略虚假信号

Targeted Activation Penalties Help CNNs Ignore Spurious Signals

We investigate whether three types of post hoc model explanations--feature
attribution, concept activation, and training point ranking--are effective for
detecting a model's reliance on spurious signals in the training data.
Specifically, we consider the scenario where the spurious signal to be detected
is unknown, at test-time, to the user of the explanation method. We design an
empirical methodology that uses semi-synthetic datasets along with
pre-specified spurious artifacts to obtain models that verifiably rely on these
spurious training signals. We then provide a suite of metrics that assess an
explanation method's reliability for spurious signal detection under various
conditions. We find that the post hoc explanation methods tested are
ineffective when the spurious artifact is unknown at test-time especially for
non-visible artifacts like a background blur. Further, we find that feature
attribution methods are susceptible to erroneously indicating dependence on
spurious signals even when the model being explained does not rely on spurious
artifacts. This finding casts doubt on the utility of these approaches, in the
hands of a practitioner, for detecting a model's reliance on spurious signals.

通过使用半合成数据集和预设的假象损伤，我们设计了一种经验方法，通过提供一组指标来评估解释方法在各种条件下检测虚假信号的可靠性。我们发现，当假象仅在测试时由解释方法的用户不知道时，后续解释方法测试是无效的，尤其是对于非可见的背景模糊等假象。同时我们也发现即使所解释的模型不依赖于虚假损伤，特征显著性方法也容易错误地指示对虚假信息的依赖。这一发现使得这些方法检测模型对虚假信号的依赖性的效用存在疑问。

事後解釋對檢測未知假相關可能無效

Post hoc Explanations may be Ineffective for Detecting Unknown Spurious  Correlation

In attempts to produce ML models less reliant on spurious patterns in NLP
datasets, researchers have recently proposed curating counterfactually
augmented data (CAD) via a human-in-the-loop process in which given some
documents and their (initial) labels, humans must revise the text to make a
counterfactual label applicable. Importantly, edits that are not necessary to
flip the applicable label are prohibited. Models trained on the augmented data
appear, empirically, to rely less on semantically irrelevant words and to
generalize better out of domain. While this work draws loosely on causal
thinking, the underlying causal model (even at an abstract level) and the
principles underlying the observed out-of-domain improvements remain unclear.
In this paper, we introduce a toy analog based on linear Gaussian models,
observing interesting relationships between causal models, measurement noise,
out-of-domain generalization, and reliance on spurious signals. Our analysis
provides some insights that help to explain the efficacy of CAD. Moreover, we
develop the hypothesis that while adding noise to causal features should
degrade both in-domain and out-of-domain performance, adding noise to
non-causal features should lead to relative improvements in out-of-domain
performance. This idea inspires a speculative test for determining whether a
feature attribution technique has identified the causal spans. If adding noise
(e.g., by random word flips) to the highlighted spans degrades both in-domain
and out-of-domain performance on a battery of challenge datasets, but adding
noise to the complement gives improvements out-of-domain, it suggests we have
identified causal spans. We present a large-scale empirical study comparing
spans edited to create CAD to those selected by attention and saliency maps.
Across numerous domains and models, we find that the hypothesized phenomenon is
pronounced for CAD.

本文提出了一种基于因果模型的玩具模型，探讨了因果模型、测量噪声、跨领域泛化以及对虚假信号的依赖之间的有趣关系，并以大规模经验研究为基础，比较了用于创建 CAD 的跨度与关注区域和显著性图所选跨度之间的差异，发现对于 CAD，研究中提出的假说在许多领域和模型中都是显著的。