The remarkable success of Vision Transformers in Artificial Neural Networks
(ANNs) has led to a growing interest in incorporating the self-attention
mechanism and transformer-based architecture into Spiking Neural Networks
(SNNs). While existing methods propose spiking self-attention mechanisms that
are compatible with SNNs, they lack reasonable scaling methods, and the overall
architectures proposed by these methods suffer from a bottleneck in effectively
extracting local features. To address these challenges, we propose a novel
spiking self-attention mechanism named Dual Spike Self-Attention (DSSA) with a
reasonable scaling method. Based on DSSA, we propose a novel spiking Vision
Transformer architecture called SpikingResformer, which combines the
ResNet-based multi-stage architecture with our proposed DSSA to improve both
performance and energy efficiency while reducing parameters. Experimental
results show that SpikingResformer achieves higher accuracy with fewer
parameters and lower energy consumption than other spiking Vision Transformer
counterparts. Notably, our SpikingResformer-L achieves 79.40% top-1 accuracy on
ImageNet with 4 time-steps, which is the state-of-the-art result in the SNN
field.

提出了一种名为 Dual Spike Self-Attention (DSSA) 的新型脉冲自注意机制以及相应的 Spiking Neural Networks 架构，命名为 SpikingResformer，通过将 ResNet-based 多阶段架构与 DSSA 相结合，提高了性能和能源效率，同时减少了参数。实验结果表明，相比其他脉冲式 Vision Transformer，SpikingResformer 在 ImageNet 上以 4 个时间步长获得了 79.40% 的 top-1 准确率，为该领域的最新成果。