Three important criteria of existing convolutional neural networks (CNNs) are
(1) test-set accuracy; (2) out-of-distribution accuracy; and (3)
explainability. While these criteria have been studied independently, their
relationship is unknown. For example, do CNNs that have a stronger
out-of-distribution performance have also stronger explainability? Furthermore,
most prior feature-importance studies only evaluate methods on 2-3 common
vanilla ImageNet-trained CNNs, leaving it unknown how these methods generalize
to CNNs of other architectures and training algorithms. Here, we perform the
first, large-scale evaluation of the relations of the three criteria using 9
feature-importance methods and 12 ImageNet-trained CNNs that are of 3 training
algorithms and 5 CNN architectures. We find several important insights and
recommendations for ML practitioners. First, adversarially robust CNNs have a
higher explainability score on gradient-based attribution methods (but not
CAM-based or perturbation-based methods). Second, AdvProp models, despite being
highly accurate more than both vanilla and robust models alone, are not
superior in explainability. Third, among 9 feature attribution methods tested,
GradCAM and RISE are consistently the best methods. Fourth, Insertion and
Deletion are biased towards vanilla and robust models respectively, due to
their strong correlation with the confidence score distributions of a CNN.
Fifth, we did not find a single CNN to be the best in all three criteria, which
interestingly suggests that CNNs are harder to interpret as they become more
accurate.

本文使用 9 种特征重要性方法和 12 种各种架构和培训算法的 ImageNet CNN 进行了第一次大规模评估，并发现了其中的一些重要洞察和建议，例如，GradCAM 和 RISE 是最佳方法，插入和删除分别对应于普通和稳健模型。

可解释的对抗性鲁棒卷积神经网络有多好？

How explainable are adversarially-robust CNNs?

Ensemble-based debiasing methods have been shown effective in mitigating the
reliance of classifiers on specific dataset bias, by exploiting the output of a
bias-only model to adjust the learning target. In this paper, we focus on the
bias-only model in these ensemble-based methods, which plays an important role
but has not gained much attention in the existing literature. Theoretically, we
prove that the debiasing performance can be damaged by inaccurate uncertainty
estimations of the bias-only model. Empirically, we show that existing
bias-only models fall short in producing accurate uncertainty estimations.
Motivated by these findings, we propose to conduct calibration on the bias-only
model, thus achieving a three-stage ensemble-based debiasing framework,
including bias modeling, model calibrating, and debiasing. Experimental results
on NLI and fact verification tasks show that our proposed three-stage debiasing
framework consistently outperforms the traditional two-stage one in
out-of-distribution accuracy.

本文提出了一种包括偏差建模、模型校准和去偏差三个阶段的集成去偏差框架，该框架在 NLI 和事实验证任务上的实验结果表明，相比传统的两阶段方法，该框架能够稳定提高模型在超出分布的情况下的准确性。