Bayesian neural networks (BNNs) provide a formalism to quantify and calibrate
uncertainty in deep learning. Current inference approaches for BNNs often
resort to few-sample estimation for scalability, which can harm predictive
performance, while its alternatives tend to be computationally prohibitively
expensive. We tackle this challenge by revealing a previously unseen connection
between inference on BNNs and volume computation problems. With this
observation, we introduce a novel collapsed inference scheme that performs
Bayesian model averaging using collapsed samples. It improves over a
Monte-Carlo sample by limiting sampling to a subset of the network weights
while pairing it with some closed-form conditional distribution over the rest.
A collapsed sample represents uncountably many models drawn from the
approximate posterior and thus yields higher sample efficiency. Further, we
show that the marginalization of a collapsed sample can be solved analytically
and efficiently despite the non-linearity of neural networks by leveraging
existing volume computation solvers. Our proposed use of collapsed samples
achieves a balance between scalability and accuracy. On various regression and
classification tasks, our collapsed Bayesian deep learning approach
demonstrates significant improvements over existing methods and sets a new
state of the art in terms of uncertainty estimation as well as predictive
performance.

提出一种新的 Bayesian 深度学习方案，使用 collapsed samples 进行贝叶斯模型平均，从而平衡神经网络的可扩展性和准确性，并利用现有的体积计算求解器解决选取部分权重采样，从而达到提高样本效率的目的，并在各种回归和分类任务中实现了显着的改进，推动了不确定性预测和预测性能的新水平。