Explaining the decisions of an Artificial Intelligence (AI) model is increasingly critical in many real-world, high-stake applications. Hundreds of papers have either proposed new feature attribution methods, discussed or harnessed these tools in their work. However, despite humans being the target end-users, most attribution methods were only evaluated on proxy automatic-evaluation metrics. In this paper, we conduct the first, large-scale user study on 320 lay and 11 expert users to shed light on the effectiveness of state-of-the-art attribution methods in assisting humans in ImageNet classification, Stanford Dogs fine-grained classification, and these two tasks but when the input image contains adversarial perturbations. We found that, in overall, feature attribution is surprisingly not more effective than showing humans nearest training-set examples. On a hard task of fine-grained dog categorization, presenting attribution maps to humans does not help, but instead hurts the performance of human-AI teams compared to AI alone. Importantly, we found automatic attribution-map evaluation measures to correlate poorly with the actual human-AI team performance. Our findings encourage the community to rigorously test their methods on the downstream human-in-the-loop applications and to rethink the existing evaluation metrics.

本研究第一次进行用户研究来证明特征归因图对于ImageNet分类和Stanford Dogs细粒度分类的辅助作用，在更难的分类任务上，特征归因图并没有帮助，相反会降低人工智能和人类团队的性能，并且自动特征归因图评估指标与实际人工智能-人类团队的表现相关性较差，本研究提倡社区在人机互动的应用中对他们的方法进行严格的测试，并重新思考现有的评估指标。

特征归属方法的效果及其与自动评估得分的相关性