The Laplace approximation (LA) of the Bayesian posterior is a Gaussian
distribution centered at the maximum a posteriori estimate. Its appeal in
Bayesian deep learning stems from the ability to quantify uncertainty post-hoc
(i.e., after standard network parameter optimization), the ease of sampling
from the approximate posterior, and the analytic form of model evidence.
However, an important computational bottleneck of LA is the necessary step of
calculating and inverting the Hessian matrix of the log posterior. The Hessian
may be approximated in a variety of ways, with quality varying with a number of
factors including the network, dataset, and inference task. In this paper, we
propose an alternative framework that sidesteps Hessian calculation and
inversion. The Hessian-free Laplace (HFL) approximation uses curvature of both
the log posterior and network prediction to estimate its variance. Only two
point estimates are needed: the standard maximum a posteriori parameter and the
optimal parameter under a loss regularized by the network prediction. We show
that, under standard assumptions of LA in Bayesian deep learning, HFL targets
the same variance as LA, and can be efficiently amortized in a pre-trained
network. Experiments demonstrate comparable performance to that of exact and
approximate Hessians, with excellent coverage for in-between uncertainty.

在该研究中，我们提出了一种利用拉普拉斯近似的替代框架，通过使用后验的曲率和网络预测来估计方差，既避免了计算和翻转黑塞矩阵的步骤，又能够在预训练网络中高效地进行。实验证明，相比于精确和近似黑塞矩阵，该方法表现相当，并具有良好的不确定性覆盖范围。