Networks of spiking neurons underpin the extraordinary information-processing
capabilities of the brain and have emerged as pillar models in neuromorphic
intelligence. Despite extensive research on spiking neural networks (SNNs),
most are established on deterministic models. Integrating noise into SNNs leads
to biophysically more realistic neural dynamics and may benefit model
performance. This work presents the noisy spiking neural network (NSNN) and the
noise-driven learning rule (NDL) by introducing a spiking neuron model
incorporating noisy neuronal dynamics. Our approach shows how noise may act as
a resource for computation and learning and theoretically provides a framework
for general SNNs. Moreover, NDL provides an insightful rationale for surrogate
gradients. By incorporating various SNN architectures and algorithms, we show
that our approach exhibits competitive performance and improved robustness
against challenging perturbations than deterministic SNNs. Additionally, we
demonstrate the utility of the NSNN model for neural coding studies. Overall,
NSNN offers a powerful, flexible, and easy-to-use tool for machine learning
practitioners and computational neuroscience researchers.

本研究提出了噪声脉冲神经网络（NSNN）和噪声驱动学习规则（NDL），并将噪声引入到神经元动力学中，展示了噪声如何作为计算和学习的资源，并提供了一种理论框架。同时，NDL 提供了一个替代梯度的合理性解释。结合各种 SNN 体系结构和算法，我们的方法表现出了竞争性能和鲁棒性，同时还证明了 NSNN 模型在神经编码研究中的实用性。此外，NSNN 为机器学习从业者和计算神经科学研究人员提供了一个强大、灵活和易于使用的工具。

利用噪声作为脉冲神经网络中计算和学习的资源

Exploiting Noise as a Resource for Computation and Learning in Spiking  Neural Networks

Inspired by the operation of biological brains, Spiking Neural Networks
(SNNs) have the unique ability to detect information encoded in spatio-temporal
patterns of spiking signals. Examples of data types requiring spatio-temporal
processing include logs of time stamps, e.g., of tweets, and outputs of neural
prostheses and neuromorphic sensors. In this paper, the second of a series of
three review papers on SNNs, we first review models and training algorithms for
the dominant approach that considers SNNs as a Recurrent Neural Network (RNN)
and adapt learning rules based on backpropagation through time to the
requirements of SNNs. In order to tackle the non-differentiability of the
spiking mechanism, state-of-the-art solutions use surrogate gradients that
approximate the threshold activation function with a differentiable function.
Then, we describe an alternative approach that relies on probabilistic models
for spiking neurons, allowing the derivation of local learning rules via
stochastic estimates of the gradient. Finally, experiments are provided for
neuromorphic data sets, yielding insights on accuracy and convergence under
different SNN models.

本文从认知神经科学方面入手，分析了 Spiking Neural Networks 这种新型生物神经样本学习 / 数据处理模型在模型、训练算法上面的优点和缺点，其中使用了概率模型和时间反向传播算法解决了部分难点，最后在神经元数据集上进行了对比实验.

脉冲神经网络 -- 第二部分：检测时空模式

Spiking Neural Networks -- Part II: Detecting Spatio-Temporal Patterns

The Kullback-Leibler (KL) divergence is a fundamental equation of information
theory that quantifies the proximity of two probability distributions. Although
difficult to understand by examining the equation, an intuition and
understanding of the KL divergence arises from its intimate relationship with
likelihood theory. We discuss how KL divergence arises from likelihood theory
in an attempt to provide some intuition and reserve a rigorous (but rather
simple) derivation for the appendix. Finally, we comment on recent applications
of KL divergence in the neural coding literature and highlight its natural
application.

讨论了 Kullback-Leibler (KL) 散度在可能性理论中的关系，在附录中对其进行了简单严谨的推导，并强调了其在神经编码领域中的自然应用。

Kullback-Leibler 散度和似然的注记

Notes on Kullback-Leibler Divergence and Likelihood

We demonstrate that the information contained in the spike occurrence times
of a population of neurons can be broken up into a series of terms, each of
which reflect something about potential coding mechanisms. This is possible in
the coding r{\'e}gime in which few spikes are emitted in the relevant time
window.
This approach allows us to study the additional information contributed by
spike timing beyond that present in the spike counts; to examine the
contributions to the whole information of different statistical properties of
spike trains, such as firing rates and correlation functions; and forms the
basis for a new quantitative procedure for the analysis of simultaneous
multiple neuron recordings. It also provides theoretical constraints upon
neural coding strategies. We find a transition between two coding r{\'e}gimes,
depending upon the size of the relevant observation timescale. For time windows
shorter than the timescale of the stimulus-induced response fluctuations, there
exists a spike count coding phase, where the purely temporal information is of
third order in time. For time windows much longer than the characteristic
timescale, there can be additional timing information of first order, leading
to a temporal coding phase in which timing information may affect the
instantaneous information rate.
We study the relative contributions of the dynamic firing rate and
correlation variables to the full temporal information; the interaction of
signal and noise correlations in temporal coding; synergy between spikes and
between cells; and the effect of refractoriness. We illustrate the utility of
the technique by analysis of a few cells from the rat barrel cortex.

本文提出了一种方法，将群体神经元的尖峰发生时间的信息分解成一系列项，每一项都反映了潜在编码机制，并为同时记录多个神经元的分析提供了理论约束。