Large-scale linear models are ubiquitous throughout machine learning, with
contemporary application as surrogate models for neural network uncertainty
quantification; that is, the linearised Laplace method. Alas, the computational
cost associated with Bayesian linear models constrains this method's
application to small networks, small output spaces and small datasets. We
address this limitation by introducing a scalable sample-based Bayesian
inference method for conjugate Gaussian multi-output linear models, together
with a matching method for hyperparameter (regularisation) selection.
Furthermore, we use a classic feature normalisation method (the g-prior) to
resolve a previously highlighted pathology of the linearised Laplace method.
Together, these contributions allow us to perform linearised neural network
inference with ResNet-18 on CIFAR100 (11M parameters, 100 outputs x 50k
datapoints), with ResNet-50 on Imagenet (50M parameters, 1000 outputs x 1.2M
datapoints) and with a U-Net on a high-resolution tomographic reconstruction
task (2M parameters, 251k output~dimensions).

引入可扩展的基于样本的 Bayesian 推断方法、匹配的超参数选择方法和经典特征归一化方法，以解决线性化神经网络推断中的计算成本限制和先前突出的路径学问题，从而在 CIFAR100、Imagenet 和高分辨率层析成像重建任务中实现线性化神经网络推断。

基于采样的大型线性模型推断，应用于线性化拉普拉斯

Sampling-based inference for large linear models, with application to linearised Laplace

In the Bayesian stochastic search variable selection framework, a common
prior distribution for the regression coefficients is the g-prior of Zellner
(1986). However, there are two standard cases in which the associated
covariance matrix does not exist, and the conventional prior of Zellner can not
be used: if the number of observations is lower than the number of variables
(large p and small n paradigm), or if some variables are linear combinations of
others. In such situations a prior distribution derived from the prior of
Zellner can be used, by introducing a ridge parameter. This prior introduced by
Gupta and Ibrahim (2007) is a flexible and simple adaptation of the g-prior. In
this paper we study the influence of the ridge parameter on the selection of
variables. A simple way to choose the associated hyper-parameters is proposed.
The method is valid for any generalized linear mixed model and we focus on the
case of probit mixed models when some variables are linear combinations of
others. The method is applied to both simulated and real datasets obtained from
Affymetrix microarray experiments. Results are compared to those obtained with
the Bayesian Lasso.

本文介绍了在贝叶斯随机搜索变量选择框架下，如何通过引入岭参数的灵活简单适配 g-prior 的方法，选择变量及超参数，并与贝叶斯 Lasso 进行了比较。