Bayesian Neural Networks (BNN) have emerged as a crucial approach for
interpreting ML predictions. By sampling from the posterior distribution, data
scientists may estimate the uncertainty of an inference. Unfortunately many
inference samples are often needed, the overhead of which greatly hinder BNN's
wide adoption. To mitigate this, previous work proposed propagating the first
and second moments of the posterior directly through the network. However, on
its own this method is even slower than sampling, so the propagated variance
needs to be approximated such as assuming independence between neural nodes.
The resulting trade-off between quality and inference time did not match even
plain Monte Carlo sampling.
Our contribution is a more principled variance propagation framework based on
"spiked covariance matrices", which smoothly interpolates between quality and
inference time. This is made possible by a new fast algorithm for updating a
diagonal-plus-low-rank matrix approximation under various operations. We tested
our algorithm against sampling based MC Dropout and Variational Inference on a
number of downstream uncertainty themed tasks, such as calibration and
out-of-distribution testing. We find that Favour is as fast as performing 2-3
inference samples, while matching the performance of 10-100 samples.
In summary, this work enables the use of BNN in the realm of performance
critical tasks where they have previously been out of reach.

本研究提出了一种基于 “尖峰协方差矩阵” 的更加原理性的方差传播框架，可在质量和推理时间之间平滑插值，使得贝叶斯神经网络能够在之前无法触及的性能关键任务领域中使用。

Favour：一种用于确定不确定性评级的快速方差算子

Favour: FAst Variance Operator for Uncertainty Rating

In recent years, newly developed methods to train spiking neural networks
(SNNs) have rendered them as a plausible alternative to Artificial Neural
Networks (ANNs) in terms of accuracy, while at the same time being much more
energy efficient at inference and potentially at training time. However, it is
still unclear what constitutes a good initialisation for an SNN. We often use
initialisation schemes developed for ANN training which are often inadequate
and require manual tuning. In this paper, we attempt to tackle this issue by
using techniques from the ANN initialisation literature as well as
computational neuroscience results. We show that the problem of weight
initialisation for ANNs is a more nuanced problem than it is for ANNs due to
the spike-and-reset non-linearity of SNNs and the firing rate collapse problem.
We firstly identify and propose several solutions to the firing rate collapse
problem under different sets of assumptions which successfully solve the issue
by leveraging classical random walk and Wiener processes results. Secondly, we
devise a general strategy for SNN initialisation which combines variance
propagation techniques from ANNs and different methods to obtain the expected
firing rate and membrane potential distribution based on diffusion and
shot-noise approximations. Altogether, we obtain theoretical results to solve
the SNN initialisation which consider the membrane potential distribution in
the presence of a threshold. Yet, to what extent can these methods be
successfully applied to SNNs on real datasets remains an open question.

本文提出一种解决 spiking neural networks 初始权重选择的方法，其中结合了 ANN 的方差传播技术和扩散和 shot-noise 逼近的方法来获得预期的放电率和膜电位分布，并成功解决了射频崩溃问题。