Spiking neural networks (SNNs) are well known as the brain-inspired models
with high computing efficiency, due to a key component that they utilize spikes
as information units, close to the biological neural systems. Although spiking
based models are energy efficient by taking advantage of discrete spike
signals, their performance is limited by current network structures and their
training methods. As discrete signals, typical SNNs cannot apply the gradient
descent rules directly into parameters adjustment as artificial neural networks
(ANNs). Aiming at this limitation, here we propose a novel method of
constructing deep SNN models with knowledge distillation (KD) that uses ANN as
teacher model and SNN as student model. Through ANN-SNN joint training
algorithm, the student SNN model can learn rich feature information from the
teacher ANN model through the KD method, yet it avoids training SNN from
scratch when communicating with non-differentiable spikes. Our method can not
only build a more efficient deep spiking structure feasibly and reasonably, but
use few time steps to train whole model compared to direct training or ANN to
SNN methods. More importantly, it has a superb ability of noise immunity for
various types of artificial noises and natural signals. The proposed novel
method provides efficient ways to improve the performance of SNN through
constructing deeper structures in a high-throughput fashion, with potential
usage for light and efficient brain-inspired computing of practical scenarios.

本文提出了一种新的方法，利用知识蒸馏构建深度 SNN 模型，通过利用 ANN 作为教师模型和 SNN 作为学生模型进行联合训练算法，可以更加有效地构建深层脉冲神经网络结构，并且在与不可微分波形的通信时，仍然能够避免从零开始训练 SNN，并且具有良好的噪声免疫能力。