Meta-learning is a framework in which machine learning models train over a
set of datasets in order to produce predictions on new datasets at test time.
Probabilistic meta-learning has received an abundance of attention from the
research community in recent years, but a problem shared by many existing
probabilistic meta-models is that they require a very large number of datasets
in order to produce high-quality predictions with well-calibrated uncertainty
estimates. In many applications, however, such quantities of data are simply
not available.
In this dissertation we present a significantly more data-efficient approach
to probabilistic meta-learning through per-datapoint amortisation of inference
in Bayesian neural networks, introducing the Amortised Pseudo-Observation
Variational Inference Bayesian Neural Network (APOVI-BNN). First, we show that
the approximate posteriors obtained under our amortised scheme are of similar
or better quality to those obtained through traditional variational inference,
despite the fact that the amortised inference is performed in a single forward
pass. We then discuss how the APOVI-BNN may be viewed as a new member of the
neural process family, motivating the use of neural process training objectives
for potentially better predictive performance on complex problems as a result.
Finally, we assess the predictive performance of the APOVI-BNN against other
probabilistic meta-models in both a one-dimensional regression problem and in a
significantly more complex image completion setting. In both cases, when the
amount of training data is limited, our model is the best in its class.

在本文中，我们提出了一种更数据高效的概率元学习方法，通过对贝叶斯神经网络的推理进行逐数据点摊销，引入了摊销伪观测变分推理贝叶斯神经网络（APOVI-BNN）。我们展示了在我们的摊销方案下获得的近似后验与传统变分推理获得的近似后验在质量上是相似或更好的，尽管摊销推理是在单个前向传递中执行的。然后，我们讨论了如何将 APOVI-BNN 看作神经过程家族的新成员，以此作为激励使用神经过程训练目标，以实现在复杂问题上更好的预测性能。最后，我们评估了 APOVI-BNN 在一维回归问题和更复杂的图像完成设置中与其他概率元模型的预测性能，在训练数据有限的情况下，我们的模型在同类模型中表现最好。

贝叶斯神经网络中的摊还推断

Amortised Inference in Bayesian Neural Networks

Domain generalization models learn to generalize to previously unseen
domains, but suffer from prediction uncertainty and domain shift. In this
paper, we address both problems. We introduce a probabilistic meta-learning
model for domain generalization, in which classifier parameters shared across
domains are modeled as distributions. This enables better handling of
prediction uncertainty on unseen domains. To deal with domain shift, we learn
domain-invariant representations by the proposed principle of meta variational
information bottleneck, we call MetaVIB. MetaVIB is derived from novel
variational bounds of mutual information, by leveraging the meta-learning
setting of domain generalization. Through episodic training, MetaVIB learns to
gradually narrow domain gaps to establish domain-invariant representations,
while simultaneously maximizing prediction accuracy. We conduct experiments on
three benchmarks for cross-domain visual recognition. Comprehensive ablation
studies validate the benefits of MetaVIB for domain generalization. The
comparison results demonstrate our method outperforms previous approaches
consistently.

本文介绍了一种面向领域概括的概率元学习模型，并通过提出的元变分信息瓶颈原则，即 MetaVIB，学习了领域不变表示，从而更好地处理了预测不确定性和领域转移问题。