Attention module does not always help deep models learn causal features that
are robust in any confounding context, e.g., a foreground object feature is
invariant to different backgrounds. This is because the confounders trick the
attention to capture spurious correlations that benefit the prediction when the
training and testing data are IID (identical & independent distribution); while
harm the prediction when the data are OOD (out-of-distribution). The sole
fundamental solution to learn causal attention is by causal intervention, which
requires additional annotations of the confounders, e.g., a "dog" model is
learned within "grass+dog" and "road+dog" respectively, so the "grass" and
"road" contexts will no longer confound the "dog" recognition. However, such
annotation is not only prohibitively expensive, but also inherently
problematic, as the confounders are elusive in nature. In this paper, we
propose a causal attention module (CaaM) that self-annotates the confounders in
unsupervised fashion. In particular, multiple CaaMs can be stacked and
integrated in conventional attention CNN and self-attention Vision Transformer.
In OOD settings, deep models with CaaM outperform those without it
significantly; even in IID settings, the attention localization is also
improved by CaaM, showing a great potential in applications that require robust
visual saliency. Codes are available at https://github.com/Wangt-CN/CaaM.

本文提出了一种因果关系注意力模块，通过自我注释来识别难以预测的混淆变量，从而使深度学习模型更好地处理在分布不同的情况下仍具备鲁棒性的因果特征，不仅解决了监督注释需要高昂成本、混淆变量难以抽象提取等问题，并在相应实验中得到了卓越的结果。