Predictive uncertainty quantification is crucial for reliable decision-making
in various applied domains. Bayesian neural networks offer a powerful framework
for this task. However, defining meaningful priors and ensuring computational
efficiency remain significant challenges, especially for complex real-world
applications. This paper addresses these challenges by proposing a novel neural
adaptive empirical Bayes (NA-EB) framework. NA-EB leverages a class of implicit
generative priors derived from low-dimensional distributions. This allows for
efficient handling of complex data structures and effective capture of
underlying relationships in real-world datasets. The proposed NA-EB framework
combines variational inference with a gradient ascent algorithm. This enables
simultaneous hyperparameter selection and approximation of the posterior
distribution, leading to improved computational efficiency. We establish the
theoretical foundation of the framework through posterior and classification
consistency. We demonstrate the practical applications of our framework through
extensive evaluations on a variety of tasks, including the two-spiral problem,
regression, 10 UCI datasets, and image classification tasks on both MNIST and
CIFAR-10 datasets. The results of our experiments highlight the superiority of
our proposed framework over existing methods, such as sparse variational
Bayesian and generative models, in terms of prediction accuracy and uncertainty
quantification.

本文提出了一种新颖的神经自适应经验贝叶斯（NA-EB）框架，该框架结合了变分推断和梯度上升算法，以实现同步超参数选择和后验分布的近似，从而提高计算效率，并通过对各种任务的广泛评估展示了我们提出框架在预测准确性和不确定性量化方面的优越性。

贝叶斯神经网络的隐式生成先验

Implicit Generative Prior for Bayesian Neural Networks

Bayesian formulations of deep learning have been shown to have compelling
theoretical properties and offer practical functional benefits, such as
improved predictive uncertainty quantification and model selection. The Laplace
approximation (LA) is a classic, and arguably the simplest family of
approximations for the intractable posteriors of deep neural networks. Yet,
despite its simplicity, the LA is not as popular as alternatives like
variational Bayes or deep ensembles. This may be due to assumptions that the LA
is expensive due to the involved Hessian computation, that it is difficult to
implement, or that it yields inferior results. In this work we show that these
are misconceptions: we (i) review the range of variants of the LA including
versions with minimal cost overhead; (ii) introduce "laplace", an easy-to-use
software library for PyTorch offering user-friendly access to all major flavors
of the LA; and (iii) demonstrate through extensive experiments that the LA is
competitive with more popular alternatives in terms of performance, while
excelling in terms of computational cost. We hope that this work will serve as
a catalyst to a wider adoption of the LA in practical deep learning, including
in domains where Bayesian approaches are not typically considered at the
moment.

本文介绍了一种叫做 Laplace approximation (LA) 的 Bayesian 神经网络逼近算法，该算法可以实现更好的不确定性估计和模型选择，并通过实验证明其在计算成本上具有优势。

拉普拉斯重现 —— 无需费力的贝叶斯深度学习

Laplace Redux -- Effortless Bayesian Deep Learning

Surrogate modeling and uncertainty quantification tasks for PDE systems are
most often considered as supervised learning problems where input and output
data pairs are used for training. The construction of such emulators is by
definition a small data problem which poses challenges to deep learning
approaches that have been developed to operate in the big data regime. Even in
cases where such models have been shown to have good predictive capability in
high dimensions, they fail to address constraints in the data implied by the
PDE model. This paper provides a methodology that incorporates the governing
equations of the physical model in the loss/likelihood functions. The resulting
physics-constrained, deep learning models are trained without any labeled data
(e.g. employing only input data) and provide comparable predictive responses
with data-driven models while obeying the constraints of the problem at hand.
This work employs a convolutional encoder-decoder neural network approach as
well as a conditional flow-based generative model for the solution of PDEs,
surrogate model construction, and uncertainty quantification tasks. The
methodology is posed as a minimization problem of the reverse Kullback-Leibler
(KL) divergence between the model predictive density and the reference
conditional density, where the later is defined as the Boltzmann-Gibbs
distribution at a given inverse temperature with the underlying potential
relating to the PDE system of interest. The generalization capability of these
models to out-of-distribution input is considered. Quantification and
interpretation of the predictive uncertainty is provided for a number of
problems.

提出一种基于物理约束深度学习的建模和不确定性量化方法，避免深度学习在小样本问题上的跨度不足，可以用于偏微分方程系统的解决和预测推断，并提供一些解释和量化手段。