Recent advances in depthwise-separable convolutional neural networks
(DS-CNNs) have led to novel architectures, that surpass the performance of
classical CNNs, by a considerable scalability and accuracy margin. This paper
reveals another striking property of DS-CNN architectures: discernible and
explainable patterns emerge in their trained depthwise convolutional kernels in
all layers. Through an extensive analysis of millions of trained filters, with
different sizes and from various models, we employed unsupervised clustering
with autoencoders, to categorize these filters. Astonishingly, the patterns
converged into a few main clusters, each resembling the difference of Gaussian
(DoG) functions, and their first and second-order derivatives. Notably, we were
able to classify over 95\% and 90\% of the filters from state-of-the-art
ConvNextV2 and ConvNeXt models, respectively. This finding is not merely a
technological curiosity; it echoes the foundational models neuroscientists have
long proposed for the vision systems of mammals. Our results thus deepen our
understanding of the emergent properties of trained DS-CNNs and provide a
bridge between artificial and biological visual processing systems. More
broadly, they pave the way for more interpretable and biologically-inspired
neural network designs in the future.

最近在深度可分离卷积神经网络（DS-CNNs）方面的进展已经引发了新的架构，通过相当大的可扩展性和准确性优势，超越了经典 CNNs 的性能。本文揭示了 DS-CNN 架构的另一个引人注目的特性：它们训练后的深度卷积核在所有层中出现了明显可辨和可解释的模式。通过对数百万个训练滤波器的广泛分析，我们使用自编码器进行无监督聚类，对这些滤波器进行了分类。令人惊讶的是，这些模式汇聚成了几个主要簇，每个簇都类似于高斯差分函数（DoG）及其一阶和二阶导数。值得注意的是，我们能够将来自最先进的 ConvNextV2 和 ConvNeXt 模型的滤波器分别分类为 95％和 90％以上。这一发现不仅仅是技术上的奇特之处；它回应了神经科学家长期以来对哺乳动物视觉系统的基础模型的提议。因此，我们的结果加深了对训练后 DS-CNN 的出现属性的理解，并在人工和生物视觉处理系统之间提供了一个桥梁。更广泛地说，它们为未来更具可解释性和生物灵感的神经网络设计铺平了道路。