Spiking Neural Networks (SNN). SNNs are based on a more biologically inspired
approach than usual artificial neural networks. Such models are characterized
by complex dynamics between neurons and spikes. These are very sensitive to the
hyperparameters, making their optimization challenging. To tackle
hyperparameter optimization of SNNs, we initially extended the signal loss
issue of SNNs to what we call silent networks. These networks fail to emit
enough spikes at their outputs due to mistuned hyperparameters or architecture.
Generally, search spaces are heavily restrained, sometimes even discretized, to
prevent the sampling of such networks. By defining an early stopping criterion
detecting silent networks and by designing specific constraints, we were able
to instantiate larger and more flexible search spaces. We applied a constrained
Bayesian optimization technique, which was asynchronously parallelized, as the
evaluation time of a SNN is highly stochastic. Large-scale experiments were
carried-out on a multi-GPU Petascale architecture. By leveraging silent
networks, results show an acceleration of the search, while maintaining good
performances of both the optimization algorithm and the best solution obtained.
We were able to apply our methodology to two popular training algorithms, known
as spike timing dependent plasticity and surrogate gradient. Early detection
allowed us to prevent worthless and costly computation, directing the search
toward promising hyperparameter combinations. Our methodology could be applied
to multi-objective problems, where the spiking activity is often minimized to
reduce the energy consumption. In this scenario, it becomes essential to find
the delicate frontier between low-spiking and silent networks. Finally, our
approach may have implications for neural architecture search, particularly in
defining suitable spiking architectures.

通过检测静默网络并设计特定约束条件，我们能够实例化更大更灵活的搜索空间，从而加速 SNN 的搜索，并保持优化算法和最佳解决方案的良好性能。我们的方法还可以用于多目标问题，其中尽可能减少脉冲活动以降低能耗是常见的。在这种情况下，找到低脉冲和静默网络之间的微妙平衡至关重要。最后，我们的方法可能对神经架构搜索产生影响，特别是在定义合适的脉冲架构方面。

脉冲神经网络的并行超参数优化

Parallel Hyperparameter Optimization Of Spiking Neural Network

Artificial Neural Networks (ANN) have gained large popularity thanks to their
ability to learn using the well-known backpropagation algorithm. On the other
hand, Spiking Neural Networks (SNNs), despite having wider abilities than ANNs,
have always presented a challenge in the training phase. This paper shows a new
method to perform supervised learning on SNNs, using Spike Timing Dependent
Plasticity (STDP) and homeostasis, aiming at training the network to identify
spatial patterns. The method is tested using the SpiNNaker digital
architecture. A SNN is trained to recognise one or multiple patterns and
performance metrics are extracted to measure the performance of the network.
Some considerations are drawn from the results showing that, in the case of a
single trained pattern, the network behaves as the ideal detector, with 100%
accuracy in detecting the trained pattern. However, as the number of trained
patterns on a single network increases, the accuracy of the identification is
linked to the similarities between these patterns. This method of training an
SNN to detect spatial patterns may be applied on pattern recognition in static
images or traffic analysis in computer networks, where each network packet
represents a spatial pattern. It will be stipulated that the homeostatic factor
may enable the network to detect patterns with some degree of similarities,
rather than only perfectly matching patterns.

该论文介绍了一种在脉冲神经网络（SNN）上执行监督学习的新方法，通过使用时序相关可塑性（STDP）和稳态调节训练网络以识别空间模式，该方法通过 SpiNNaker 数字架构进行测试，结果表明 SNN 能够以高准确率识别单个训练模式，但当训练模式数量增加时，识别准确率与这些模式之间的相似性有关，该方法可以应用于静态图像的模式识别或计算机网络中的流量分析，其中每个网络数据包代表一个空间模式。