Neuromorphic computing, which exploits Spiking Neural Networks (SNNs) on
neuromorphic chips, is a promising energy-efficient alternative to traditional
AI. CNN-based SNNs are the current mainstream of neuromorphic computing. By
contrast, no neuromorphic chips are designed especially for Transformer-based
SNNs, which have just emerged, and their performance is only on par with
CNN-based SNNs, offering no distinct advantage. In this work, we propose a
general Transformer-based SNN architecture, termed as ``Meta-SpikeFormer",
whose goals are: 1) Lower-power, supports the spike-driven paradigm that there
is only sparse addition in the network; 2) Versatility, handles various vision
tasks; 3) High-performance, shows overwhelming performance advantages over
CNN-based SNNs; 4) Meta-architecture, provides inspiration for future
next-generation Transformer-based neuromorphic chip designs. Specifically, we
extend the Spike-driven Transformer in \citet{yao2023spike} into a meta
architecture, and explore the impact of structure, spike-driven self-attention,
and skip connection on its performance. On ImageNet-1K, Meta-SpikeFormer
achieves 80.0\% top-1 accuracy (55M), surpassing the current state-of-the-art
(SOTA) SNN baselines (66M) by 3.7\%. This is the first direct training SNN
backbone that can simultaneously supports classification, detection, and
segmentation, obtaining SOTA results in SNNs. Finally, we discuss the
inspiration of the meta SNN architecture for neuromorphic chip design. Source
code and models are available at
https://github.com/BICLab/Spike-Driven-Transformer-V2.

通过提出一种名为 "Meta-SpikeFormer" 的通用 Transformer 架构，本论文实现了对 CNN-based SNNs 具有压倒性性能优势的低功耗、多功能、高性能、面向未来下一代神经形态芯片设计的 Meta 架构，并应用于分类、检测和分割任务中，取得了 SNN 领域的最佳结果。