Deep neural networks (NNs) are known for their high-prediction performances.
However, NNs are prone to yield unreliable predictions when encountering
completely new situations without indicating their uncertainty. Bayesian
variants of NNs (BNNs), such as Monte Carlo (MC) dropout BNNs, do provide
uncertainty measures and simultaneously increase the prediction performance.
The only disadvantage of BNNs is their higher computation time during test time
because they rely on a sampling approach. Here we present a single-shot MC
dropout approximation that preserves the advantages of BNNs while being as fast
as NNs. Our approach is based on moment propagation (MP) and allows to
analytically approximate the expected value and the variance of the MC dropout
signal for commonly used layers in NNs, i.e. convolution, max pooling, dense,
softmax, and dropout layers. The MP approach can convert an NN into a BNN
without re-training given the NN has been trained with standard dropout. We
evaluate our approach on different benchmark datasets and a simulated toy
example in a classification and regression setting. We demonstrate that our
single-shot MC dropout approximation resembles the point estimate and the
uncertainty estimate of the predictive distribution that is achieved with an MC
approach, while being fast enough for real-time deployments of BNNs. We show
that using part of the saved time to combine our MP approach with deep ensemble
techniques does further improve the uncertainty measures.

我们提出了一种快速的基于矩阵传播的近似方法，将神经网络转换成贝叶斯神经网络，实现对预测分布的点估计和不确定性估计，并通过与深度集成技术相结合进一步提高不确定性测量的性能。

神经网络的单次贝叶斯逼近

Single-shot Bayesian approximation for neural networks

Deep neural networks (DNNs) are known for their high prediction performance,
especially in perceptual tasks such as object recognition or autonomous
driving. Still, DNNs are prone to yield unreliable predictions when
encountering completely new situations without indicating their uncertainty.
Bayesian variants of DNNs (BDNNs), such as MC dropout BDNNs, do provide
uncertainty measures. However, BDNNs are slow during test time because they
rely on a sampling approach. Here we present a single shot MC dropout
approximation that preserves the advantages of BDNNs without being slower than
a DNN. Our approach is to analytically approximate for each layer in a fully
connected network the expected value and the variance of the MC dropout signal.
We evaluate our approach on different benchmark datasets and a simulated toy
example. We demonstrate that our single shot MC dropout approximation resembles
the point estimate and the uncertainty estimate of the predictive distribution
that is achieved with an MC approach, while being fast enough for real-time
deployments of BDNNs.

该研究旨在通过一种单次 MC 退火的方法来提高贝叶斯变体深度神经网络的测试时间，同时保持不确定性测量的优点。经过基准测试和模拟游戏等方面的验证，该方法能够在实时部署 BDNNs 时实现预测分布的点估计和不确定性估计。