Spiking neural networks (SNNs) have attracted considerable attention for
their event-driven, low-power characteristics and high biological
interpretability. Inspired by knowledge distillation (KD), recent research has
improved the performance of the SNN model with a pre-trained teacher model.
However, additional teacher models require significant computational resources,
and it is tedious to manually define the appropriate teacher network
architecture. In this paper, we explore cost-effective self-distillation
learning of SNNs to circumvent these concerns. Without an explicit defined
teacher, the SNN generates pseudo-labels and learns consistency during
training. On the one hand, we extend the timestep of the SNN during training to
create an implicit temporal ``teacher" that guides the learning of the original
``student", i.e., the temporal self-distillation. On the other hand, we guide
the output of the weak classifier at the intermediate stage by the final output
of the SNN, i.e., the spatial self-distillation. Our temporal-spatial
self-distillation (TSSD) learning method does not introduce any inference
overhead and has excellent generalization ability. Extensive experiments on the
static image datasets CIFAR10/100 and ImageNet as well as the neuromorphic
datasets CIFAR10-DVS and DVS-Gesture validate the superior performance of the
TSSD method. This paper presents a novel manner of fusing SNNs with KD,
providing insights into high-performance SNN learning methods.

脉冲神经网络 (SNNs) 通过事件驱动、低功耗特性和高生物解释性而引起了广泛关注。本文通过知识蒸馏 (KD) 提高了预训练教师模型的 SNN 模型性能，同时探索了一种经济高效的自我蒸馏学习方法，即时间 - 空间自我蒸馏 (TSSD) 学习方法。实验证明该方法在多个数据集上具有卓越的性能，为高性能 SNN 学习方法提供了新的融合思路。