Spiking neural networks (SNNs) are potential competitors to artificial neural
networks (ANNs) due to their high energy-efficiency on neuromorphic hardware.
However, SNNs are unfolded over simulation time steps during the training
process. Thus, SNNs require much more memory than ANNs, which impedes the
training of deeper SNN models. In this paper, we propose the reversible spiking
neural network to reduce the memory cost of intermediate activations and
membrane potentials during training. Firstly, we extend the reversible
architecture along temporal dimension and propose the reversible spiking block,
which can reconstruct the computational graph and recompute all intermediate
variables in forward pass with a reverse process. On this basis, we adopt the
state-of-the-art SNN models to the reversible variants, namely reversible
spiking ResNet (RevSResNet) and reversible spiking transformer (RevSFormer).
Through experiments on static and neuromorphic datasets, we demonstrate that
the memory cost per image of our reversible SNNs does not increase with the
network depth. On CIFAR10 and CIFAR100 datasets, our RevSResNet37 and
RevSFormer-4-384 achieve comparable accuracies and consume 3.79x and 3.00x
lower GPU memory per image than their counterparts with roughly identical model
complexity and parameters. We believe that this work can unleash the memory
constraints in SNN training and pave the way for training extremely large and
deep SNNs. The code is available at this https URL

我们提出可逆脉冲神经网络以降低训练过程中中间激活和膜电位的内存成本，通过实验证明我们的可逆 SNN 网络在网络深度上不增加每张图片的内存成本，并比现有模型在准确性和消耗的 GPU 内存方面表现更好。

高效内存可逆脉冲神经网络

Memory-Efficient Reversible Spiking Neural Networks

While backpropagation (BP) has been applied to spiking neural networks (SNNs)
achieving encouraging results, a key challenge involved is to backpropagate a
continuous-valued loss over layers of spiking neurons exhibiting discontinuous
all-or-none firing activities. Existing methods deal with this difficulty by
introducing compromises that come with their own limitations, leading to
potential performance degradation. We propose a novel BP-like method, called
neighborhood aggregation (NA), which computes accurate error gradients guiding
weight updates that may lead to discontinuous modifications of firing
activities. NA achieves this goal by aggregating finite differences of the loss
over multiple perturbed membrane potential waveforms in the neighborhood of the
present membrane potential of each neuron while utilizing a new membrane
potential distance function. Our experiments show that the proposed NA
algorithm delivers the state-of-the-art performance for SNN training on several
datasets.

提出了一种新颖的类反向传播算法 NA，用于解决在脉冲神经网络中通过邻域聚合的方式计算出精确误差梯度，进而训练模型，该算法在多个数据集上达到了目前最佳性能。