Deep neural networks (DNNs) have achieved impressive predictive performance
due to their ability to learn complex, non-linear relationships between
variables. However, the inability to effectively visualize these relationships
has led to DNNs being characterized as black boxes and consequently limited
their applications. To ameliorate this problem, we introduce the use of
hierarchical interpretations to explain DNN predictions through our proposed
method, agglomerative contextual decomposition (ACD). Given a prediction from a
trained DNN, ACD produces a hierarchical clustering of the input features,
along with the contribution of each cluster to the final prediction. This
hierarchy is optimized to identify clusters of features that the DNN learned
are predictive. Using examples from Stanford Sentiment Treebank and ImageNet,
we show that ACD is effective at diagnosing incorrect predictions and
identifying dataset bias. Through human experiments, we demonstrate that ACD
enables users both to identify the more accurate of two DNNs and to better
trust a DNN's outputs. We also find that ACD's hierarchy is largely robust to
adversarial perturbations, implying that it captures fundamental aspects of the
input and ignores spurious noise.

引入聚合上下文分解的方法来解释 DNN 预测结果。该方法通过产生输入特征的层次聚类以及每个聚类对于最终预测的贡献，实现了对 DNN 的预测结果进行解释，有效诊断不正确的预测和识别数据集的偏倚。同时人体实验也表明，ACD 能够使用户较好地识别两个 DNN 中更准确的那一个，并更加信任 DNN 的输出。