Spiking Neural Networks (SNNs) are recognized as the candidate for the
next-generation neural networks due to their bio-plausibility and energy
efficiency. Recently, researchers have demonstrated that SNNs are able to
achieve nearly state-of-the-art performance in image recognition tasks using
surrogate gradient training. However, some essential questions exist pertaining
to SNNs that are little studied: Do SNNs trained with surrogate gradient learn
different representations from traditional Artificial Neural Networks (ANNs)?
Does the time dimension in SNNs provide unique representation power? In this
paper, we aim to answer these questions by conducting a representation
similarity analysis between SNNs and ANNs using Centered Kernel Alignment
(CKA). We start by analyzing the spatial dimension of the networks, including
both the width and the depth. Furthermore, our analysis of residual connections
shows that SNNs learn a periodic pattern, which rectifies the representations
in SNNs to be ANN-like. We additionally investigate the effect of the time
dimension on SNN representation, finding that deeper layers encourage more
dynamics along the time dimension. We also investigate the impact of input data
such as event-stream data and adversarial attacks. Our work uncovers a host of
new findings of representations in SNNs. We hope this work will inspire future
research to fully comprehend the representation power of SNNs. Code is released
at this https URL

本文对比分析了脉冲神经网络和人工神经网络在表示相似性方面的表现，发现通过 CKA 的分析可以发现脉冲神经网络有类似于 ANN 的周期模式，并且深层次的脉冲神经网络对时间维度的动态表示更加有效。