Interest in spiking neural networks (SNNs) has been growing steadily,
promising an energy-efficient alternative to formal neural networks (FNNs),
commonly known as artificial neural networks (ANNs). Despite increasing
interest, especially for Edge applications, these event-driven neural networks
suffered from their difficulty to be trained compared to FNNs. To alleviate
this problem, a number of innovative methods have been developed to provide
performance more or less equivalent to that of FNNs. However, the spiking
activity of a network during inference is usually not considered. While SNNs
may usually have performance comparable to that of FNNs, it is often at the
cost of an increase of the network's activity, thus limiting the benefit of
using them as a more energy-efficient solution.
In this paper, we propose to leverage Knowledge Distillation (KD) for SNNs
training with surrogate gradient descent in order to optimize the trade-off
between performance and spiking activity. Then, after understanding why KD led
to an increase in sparsity, we also explored Activations regularization and
proposed a novel method with Logits Regularization. These approaches, validated
on several datasets, clearly show a reduction in network spiking activity
(-26.73% on GSC and -14.32% on CIFAR-10) while preserving accuracy.

利用知识蒸馏和梯度下降进行训练可以优化脉冲神经网络的性能和脉冲活动之间的平衡，同时激活正则化和逻辑回归方法可以减少网络的脉冲活动。