Bayesian neural networks and deep ensembles represent two modern paradigms of
uncertainty quantification in deep learning. Yet these approaches struggle to
scale mainly due to memory inefficiency issues, since they require parameter
storage several times higher than their deterministic counterparts. To address
this, we augment the weight matrix of each layer with a small number of
inducing weights, thereby projecting the uncertainty quantification into such
low dimensional spaces. We further extend Matheron's conditional Gaussian
sampling rule to enable fast weight sampling, which enables our inference
method to maintain reasonable run-time as compared with ensembles. Importantly,
our approach achieves competitive performance to the state-of-the-art in
prediction and uncertainty estimation tasks with fully connected neural
networks and ResNets, while reducing the parameter size to $\leq 24.3\%$ of
that of a $single$ neural network.

本研究提出一种基于感知器的设计方案，该方案结合了贝叶斯神经网络和深度集成等现代方法，通过在每层的权重矩阵中加入少量的诱导权重来降低存储和计算成本，同时保持较好的预测精度和不确定性估计能力。