We propose Radial Bayesian Neural Networks: a variational distribution for mean field variational inference (MFVI) in Bayesian neural networks that is simple to implement, scalable to large models, and robust to hyperparameter selection. We hypothesize that standard MFVI fails in large models because of a property of the high-dimensional Gaussians used as posteriors. As variances grow, samples come almost entirely from a `soap-bubble' far from the mean. We show that the ad-hoc tweaks used previously in the literature to get MFVI to work served to stop such variances growing. Designing a new posterior distribution, we avoid this pathology in a theoretically principled way. Our distribution improves accuracy and uncertainty over standard MFVI, while scaling to large data where most other VI and MCMC methods struggle. We benchmark Radial BNNs in a real-world task of diabetic retinopathy diagnosis from fundus images, a task with ~100x larger input dimensionality and model size compared to previous demonstrations of MFVI.

我们提出了径向贝叶斯神经网络（Radial Bayesian Neural Networks，简称Radial BNNs）:一种变分后验的BNNs，它适用于大型模型而仍然保持对权重空间的完全支持的分布。与其他可扩展的贝叶斯深度学习方法不同，如MC dropout或深度集合，Radial BNNs的全面支持使它们适用于顺序推理的先验。此外，它们解决了具有离散支持的权重分布的先验不合理性的概念性挑战。通过使用Radial BNNs作为相对于MFVI(稳定均场方差近似)的一种理论上合理、稳健的替代方案，我们在贝叶斯连续学习评估中取得了重大进展。

基于径向函数的贝叶斯神经网络: 在大规模贝叶斯深度学习中突破离散支持限制