Modern deep learning models are notoriously opaque, which has motivated the
development of methods for interpreting how deep models predict. This goal is
usually approached with attribution method, which assesses the influence of
features on model predictions. As an explanation method, the evaluation
criteria of attribution methods is how accurately it re-reflects the actual
reasoning process of the model (faithfulness). Meanwhile, since the reasoning
process of deep models is inaccessible, researchers design various evaluation
methods to demonstrate their arguments. However, some crucial logic traps in
these evaluation methods are ignored in most works, causing inaccurate
evaluation and unfair comparison. This paper systematically reviews existing
methods for evaluating attribution scores and summarizes the logic traps in
these methods. We further conduct experiments to demonstrate the existence of
each logic trap. Through both the theoretical and experimental analysis, we
hope to increase attention on the inaccurate evaluation of attribution scores.
Moreover, with this paper, we suggest stopping focusing on improving
performance under unreliable evaluation systems and starting efforts on
reducing the impact of proposed logic traps

该论文系统地回顾了现有的评估归属分数的方法，并总结了这些方法中的逻辑陷阱，旨在提高人们对不准确评估归属分数的注意力，并建议停止关注在不可靠的评估系统下提高性能方面的努力，而是致力于减少逻辑陷阱的影响。