Deep neural networks have been well-known for their superb handling of
various machine learning and artificial intelligence tasks. However, due to
their over-parameterized black-box nature, it is often difficult to understand
the prediction results of deep models. In recent years, many interpretation
tools have been proposed to explain or reveal how deep models make decisions.
In this paper, we review this line of research and try to make a comprehensive
survey. Specifically, we first introduce and clarify two basic concepts --
interpretations and interpretability -- that people usually get confused about.
To address the research efforts in interpretations, we elaborate the designs of
a number of interpretation algorithms, from different perspectives, by
proposing a new taxonomy. Then, to understand the interpretation results, we
also survey the performance metrics for evaluating interpretation algorithms.
Further, we summarize the current works in evaluating models' interpretability
using "trustworthy" interpretation algorithms. Finally, we review and discuss
the connections between deep models' interpretations and other factors, such as
adversarial robustness and learning from interpretations, and we introduce
several open-source libraries for interpretation algorithms and evaluation
approaches.

本文综述了神经网络的解释工具和算法，提出了一种新的分类方法，介绍了解释结果的评估方法和信任度算法的应用，讨论了深度模型解释与鲁棒性和借鉴解释的联系，并介绍了一些开源库。

可解释的深度学习：解释、可解释性、可信度及其他

Interpretable Deep Learning: Interpretation, Interpretability,  Trustworthiness, and Beyond

Deep neural networks have been shown to be fooled rather easily using
adversarial attack algorithms. Practical methods such as adversarial patches
have been shown to be extremely effective in causing misclassification.
However, these patches are highlighted using standard network interpretation
algorithms, thus revealing the identity of the adversary. We show that it is
possible to create adversarial patches which not only fool the prediction, but
also change what we interpret regarding the cause of the prediction. Moreover,
we introduce our attack as a controlled setting to measure the accuracy of
interpretation algorithms. We show this using extensive experiments for
Grad-CAM interpretation that transfers to occluding patch interpretation as
well. We believe our algorithms can facilitate developing more robust network
interpretation tools that truly explain the network's underlying decision
making process.

本文提出了一种对抗性噪声网络攻击方法，有效地误导了深度神经网络，同时也改变了网络决策的解释算法，并引入一种控制条件测试神经网络解释算法准确性的方法，以促进更健壮的神经网络解释工具发展。