Deep Neural Networks (DNNs) often fail in out-of-distribution scenarios. In
this paper, we introduce a tool to visualize and understand such failures. We
draw inspiration from concepts from neural electrophysiology, which are based
on inspecting the internal functioning of a neural networks by analyzing the
feature tuning and invariances of individual units. Deep Electrophysiology, in
short Deephys, provides insights of the DNN's failures in out-of-distribution
scenarios by comparative visualization of the neural activity in
in-distribution and out-of-distribution datasets. Deephys provides seamless
analyses of individual neurons, individual images, and a set of set of images
from a category, and it is capable of revealing failures due to the presence of
spurious features and novel features. We substantiate the validity of the
qualitative visualizations of Deephys thorough quantitative analyses using
convolutional and transformers architectures, in several datasets and
distribution shifts (namely, colored MNIST, CIFAR-10 and ImageNet).

本文介绍了一种基于神经电生理学概念的工具 Deep Electrophysiology（简称 Deephs），通过比较神经元活动在分布内和分布外数据集中的可视化，可提供深度神经网络失败情况的洞察力，通过定量分析表明 Deephs 的有效性，在颜色 MNIST，CIFAR-10 和 ImageNet 等数据集和分布转换下的卷积和 transformer 体系结构中得到证实。