Developments in machine learning interpretability techniques over the past
decade have provided new tools to observe the image regions that are most
informative for classification and localization in artificial neural networks
(ANNs). Are the same regions similarly informative to human observers? Using
data from 79 new experiments and 7,810 participants, we show that passive
attention techniques reveal a significant overlap with human visual selectivity
estimates derived from 6 distinct behavioral tasks including visual
discrimination, spatial localization, recognizability, free-viewing,
cued-object search, and saliency search fixations. We find that input
visualizations derived from relatively simple ANN architectures probed using
guided backpropagation methods are the best predictors of a shared component in
the joint variability of the human measures. We validate these correlational
results with causal manipulations using recognition experiments. We show that
images masked with ANN attention maps were easier for humans to classify than
control masks in a speeded recognition experiment. Similarly, we find that
recognition performance in the same ANN models was likewise influenced by
masking input images using human visual selectivity maps. This work contributes
a new approach to evaluating the biological and psychological validity of
leading ANNs as models of human vision: by examining their similarities and
differences in terms of their visual selectivity to the information contained
in images.

通过 79 个实验和 7,810 个实验参与者的数据研究，揭示了同人类视觉选择性估计有显著重叠的被动关注技术，指出使用引导反向传播方法探测相对简单的神经网络架构产生的输入可视化是预测共享成分的最佳选择，这些结果帮助我们通过比较不同模型在图像信息选择性方面的相似性和差异性来评估领先的神经网络模型作为人类视觉模型的生物学和心理学的有效性。