Attribution methods can provide powerful insights into the reasons for a
classifier's decision. We argue that a key desideratum of an explanation method
is its robustness to input hyperparameters which are often randomly set or
empirically tuned. High sensitivity to arbitrary hyperparameter choices does
not only impede reproducibility but also questions the correctness of an
explanation and impairs the trust of end-users. In this paper, we provide a
thorough empirical study on the sensitivity of existing attribution methods. We
found an alarming trend that many methods are highly sensitive to changes in
their common hyperparameters e.g. even changing a random seed can yield a
different explanation! Interestingly, such sensitivity is not reflected in the
average explanation accuracy scores over the dataset as commonly reported in
the literature. In addition, explanations generated for robust classifiers
(i.e. which are trained to be invariant to pixel-wise perturbations) are
surprisingly more robust than those generated for regular classifiers.

本文提供了关于现有归因方法敏感性的实证研究结果，发现许多方法对其常见超参数异常敏感，这不仅影响了研究结果的复制性，也会影响最终用户的信任度。同时，现在常见的准确性评估方法并不能完全反映出这种敏感性，并且在对抗像素扰动下训练的健壮分类器提供的解释比常规分类器更加鲁棒。