Spiking neural networks (SNNs) well support spatiotemporal learning and
energy-efficient event-driven hardware neuromorphic processors. As an important
class of SNNs, recurrent spiking neural networks (RSNNs) possess great
computational power. However, the practical application of RSNNs is severely
limited by challenges in training. Biologically-inspired unsupervised learning
has limited capability in boosting the performance of RSNNs. On the other hand,
existing backpropagation (BP) methods suffer from high complexity of unrolling
in time, vanishing and exploding gradients, and approximate differentiation of
discontinuous spiking activities when applied to RSNNs. To enable supervised
training of RSNNs under a well-defined loss function, we present a novel
Spike-Train level RSNNs Backpropagation (ST-RSBP) algorithm for training deep
RSNNs. The proposed ST-RSBP directly computes the gradient of a rated-coded
loss function defined at the output layer of the network w.r.t tunable
parameters. The scalability of ST-RSBP is achieved by the proposed spike-train
level computation during which temporal effects of the SNN is captured in both
the forward and backward pass of BP. Our ST-RSBP algorithm can be broadly
applied to RSNNs with a single recurrent layer or deep RSNNs with multiple
feed-forward and recurrent layers. Based upon challenging speech and image
datasets including TI46, N-TIDIGITS, Fashion-MNIST and MNIST, ST-RSBP is able
to train RSNNs with an accuracy surpassing that of the current state-of-art SNN
BP algorithms and conventional non-spiking deep learning models.

本文介绍了一种用于训练深度 recurrent spiking neural networks 的新算法：Spike-Train level RSNNs Backpropagation (ST-RSBP)。该算法通过直接计算网络输出层中定额编码的 loss function 相对可调参数的梯度来训练 RSNNs，在 TI46、N-TIDIGITS、Fashion-MNIST 和 MNIST 等数据集上都表现出了优于当前最先进的 SNN BP 算法和传统的非 spiking 深度学习模型的准确性。