Spiking neural networks (SNNs) have made great progress on both performance
and efficiency over the last few years,but their unique working pattern makes
it hard to train a high-performance low-latency SNN.Thus the development of
SNNs still lags behind traditional artificial neural networks (ANNs).To
compensate this gap,many extraordinary works have been
proposed.Nevertheless,these works are mainly based on the same kind of network
structure (i.e.CNN) and their performance is worse than their ANN
counterparts,which limits the applications of this http URL this end,we propose a
novel Transformer-based SNN,termed "Spikeformer",which outperforms its ANN
counterpart on both static dataset and neuromorphic dataset and may be an
alternative architecture to CNN for training high-performance SNNs.First,to
deal with the problem of "data hungry" and the unstable training period
exhibited in the vanilla model,we design the Convolutional Tokenizer (CT)
module,which improves the accuracy of the original model on DVS-Gesture by more
than 16%.Besides,in order to better incorporate the attention mechanism inside
Transformer and the spatio-temporal information inherent to SNN,we adopt
spatio-temporal attention (STA) instead of spatial-wise or temporal-wise
attention.With our proposed method,we achieve competitive or state-of-the-art
(SOTA) SNN performance on DVS-CIFAR10,DVS-Gesture,and ImageNet datasets with
the least simulation time steps (i.e.low latency).Remarkably,our Spikeformer
outperforms other SNNs on ImageNet by a large margin (i.e.more than 5%) and
even outperforms its ANN counterpart by 3.1% and 2.2% on DVS-Gesture and
ImageNet respectively,indicating that Spikeformer is a promising architecture
for training large-scale SNNs and may be more suitable for SNNs compared to
CNN.We believe that this work shall keep the development of SNNs in step with
ANNs as much as possible.Code will be available.

提出了一种基于 Transformer 的 Spikeformer 方法，它在静态数据集和神经形态数据集上均优于其人工神经网络对应物。为了更好地将注意力机制整合到 Transformer 中，并融合内在于 SNN 的时空信息，采用时空注意力机制。