Methods for understanding the decisions of and mechanisms underlying deep
neural networks (DNNs) typically rely on building intuition by emphasizing
sensory or semantic features of individual examples. For instance, methods aim
to visualize the components of an input which are "important" to a network's
decision, or to measure the semantic properties of single neurons. Here, we
argue that interpretability research suffers from an over-reliance on
intuition-based approaches that risk-and in some cases have caused-illusory
progress and misleading conclusions. We identify a set of limitations that we
argue impede meaningful progress in interpretability research, and examine two
popular classes of interpretability methods-saliency and single-neuron-based
approaches-that serve as case studies for how overreliance on intuition and
lack of falsifiability can undermine interpretability research. To address
these concerns, we propose a strategy to address these impediments in the form
of a framework for strongly falsifiable interpretability research. We encourage
researchers to use their intuitions as a starting point to develop and test
clear, falsifiable hypotheses, and hope that our framework yields robust,
evidence-based interpretability methods that generate meaningful advances in
our understanding of DNNs.

本文介绍了深度神经网络的可解释性研究及其方法，指出目前的方法存在的局限和风险，并提出了以可证伪性为核心的新的研究框架，目的是为了产生更具意义和可靠的可解释性方法。