Current approximate posteriors in Bayesian neural networks (BNNs) exhibit a
crucial limitation: they fail to maintain invariance under reparameterization,
i.e. BNNs assign different posterior densities to different parametrizations of
identical functions. This creates a fundamental flaw in the application of
Bayesian principles as it breaks the correspondence between uncertainty over
the parameters with uncertainty over the parametrized function. In this paper,
we investigate this issue in the context of the increasingly popular linearized
Laplace approximation. Specifically, it has been observed that linearized
predictives alleviate the common underfitting problems of the Laplace
approximation. We develop a new geometric view of reparametrizations from which
we explain the success of linearization. Moreover, we demonstrate that these
reparameterization invariance properties can be extended to the original neural
network predictive using a Riemannian diffusion process giving a
straightforward algorithm for approximate posterior sampling, which empirically
improves posterior fit.

Bayesian 神经网络的近似后验在重新参数化下保持不变的问题被证明在线性化拉普拉斯近似中得到缓解。通过发展一种新的几何观点来解释线性化的成功，并利用 Riemann 扩散过程将这些重新参数化不变性扩展到原始神经网络预测，从而提出了一种简单的近似后验抽样算法，从而在实证中提高了后验拟合。

近似贝叶斯推断中的重参化不变性

Reparameterization invariance in approximate Bayesian inference

The Bayesian paradigm has the potential to solve core issues of deep neural
networks such as poor calibration and data inefficiency. Alas, scaling Bayesian
inference to large weight spaces often requires restrictive approximations. In
this work, we show that it suffices to perform inference over a small subset of
model weights in order to obtain accurate predictive posteriors. The other
weights are kept as point estimates. This subnetwork inference framework
enables us to use expressive, otherwise intractable, posterior approximations
over such subsets. In particular, we implement subnetwork linearized Laplace as
a simple, scalable Bayesian deep learning method: We first obtain a MAP
estimate of all weights and then infer a full-covariance Gaussian posterior
over a subnetwork using the linearized Laplace approximation. We propose a
subnetwork selection strategy that aims to maximally preserve the model's
predictive uncertainty. Empirically, our approach compares favorably to
ensembles and less expressive posterior approximations over full networks. Our
proposed subnetwork (linearized) Laplace method is implemented within the
laplace PyTorch library at this https URL

本文介绍一种基于子网络推断框架和线性 Laplace 逼近的贝叶斯深度学习方法，用于提高深度神经网络的预测精度和有效性。在子网络中推断出如此表达力强大的后验逼近，该方法得以比全网络的后验逼近和 Ensembles 方法实现更好的效果。