Bayesian neural networks (BNN) promise to combine the predictive performance
of neural networks with principled uncertainty modeling important for
safety-critical systems and decision making. However, posterior uncertainty
estimates depend on the choice of prior, and finding informative priors in
weight-space has proven difficult. This has motivated variational inference
(VI) methods that pose priors directly on the function generated by the BNN
rather than on weights. In this paper, we address a fundamental issue with such
function-space VI approaches pointed out by Burt et al. (2020), who showed that
the objective function (ELBO) is negative infinite for most priors of interest.
Our solution builds on generalized VI (Knoblauch et al., 2019) with the
regularized KL divergence (Quang, 2019) and is, to the best of our knowledge,
the first well-defined variational objective for function-space inference in
BNNs with Gaussian process (GP) priors. Experiments show that our method
incorporates the properties specified by the GP prior on synthetic and small
real-world data sets, and provides competitive uncertainty estimates for
regression, classification and out-of-distribution detection compared to BNN
baselines with both function and weight-space priors.

Bayesian 神经网络以贝叶斯理念结合了神经网络的预测性能和对安全关键系统和决策制定至关重要的原则性不确定性建模。但是后验不确定性的估计取决于先验的选择，而在权重空间中找到信息量丰富的先验证明非常困难。为了解决这个问题，我们使用了一种基于广义 VI 的方法结合正则化的 KL 散度，可以被认为是 BNN 中具有高斯过程先验的函数空间推断的首个良定义变分目标。实验证明，我们的方法在合成数据和小型现实世界数据集上具备 GP 先验指定的特性，并与基于函数和权重空间先验的 BNN 基线相比，在回归、分类和外分布检测方面提供了有竞争力的不确定性估计。