Conditional neural processes (CNPs; Garnelo et al., 2018a) are attractive
meta-learning models which produce well-calibrated predictions and are
trainable via a simple maximum likelihood procedure. Although CNPs have many
advantages, they are unable to model dependencies in their predictions. Various
works propose solutions to this, but these come at the cost of either requiring
approximate inference or being limited to Gaussian predictions. In this work,
we instead propose to change how CNPs are deployed at test time, without any
modifications to the model or training procedure. Instead of making predictions
independently for every target point, we autoregressively define a joint
predictive distribution using the chain rule of probability, taking inspiration
from the neural autoregressive density estimator (NADE) literature. We show
that this simple procedure allows factorised Gaussian CNPs to model highly
dependent, non-Gaussian predictive distributions. Perhaps surprisingly, in an
extensive range of tasks with synthetic and real data, we show that CNPs in
autoregressive (AR) mode not only significantly outperform non-AR CNPs, but are
also competitive with more sophisticated models that are significantly more
computationally expensive and challenging to train. This performance is
remarkable given that AR CNPs are not trained to model joint dependencies. Our
work provides an example of how ideas from neural distribution estimation can
benefit neural processes, and motivates research into the AR deployment of
other neural process models.

本文介绍了一种基于条件神经过程的预测方式，在测试阶段以链式规则定义联合预测分布，从而克服条件神经过程只能独立预测的缺点，实现高度相关、非高斯的预测，且性能显著优于非自回归条件神经过程，在不需要额外的近似推断或昂贵的训练策略的情况下，达到了与更复杂的预测模型竞争的水平。

自回归条件神经过程

Autoregressive Conditional Neural Processes

Conditional Neural Processes (CNPs; Garnelo et al., 2018a) are meta-learning
models which leverage the flexibility of deep learning to produce
well-calibrated predictions and naturally handle off-the-grid and missing data.
CNPs scale to large datasets and train with ease. Due to these features, CNPs
appear well-suited to tasks from environmental sciences or healthcare.
Unfortunately, CNPs do not produce correlated predictions, making them
fundamentally inappropriate for many estimation and decision making tasks.
Predicting heat waves or floods, for example, requires modelling dependencies
in temperature or precipitation over time and space. Existing approaches which
model output dependencies, such as Neural Processes (NPs; Garnelo et al.,
2018b) or the FullConvGNP (Bruinsma et al., 2021), are either complicated to
train or prohibitively expensive. What is needed is an approach which provides
dependent predictions, but is simple to train and computationally tractable. In
this work, we present a new class of Neural Process models that make correlated
predictions and support exact maximum likelihood training that is simple and
scalable. We extend the proposed models by using invertible output
transformations, to capture non-Gaussian output distributions. Our models can
be used in downstream estimation tasks which require dependent function
samples. By accounting for output dependencies, our models show improved
predictive performance on a range of experiments with synthetic and real data.

我们提出了一种新的神经过程模型，可提供相关预测，支持精确的最大似然训练，是简单且可扩展的，并通过使用可逆输出转换来增强模型的能力，从而捕获非高斯输出分布。在合成和真实数据的一系列实验中表现出改进的预测性能，可以在需要相关函数样本的下游估计任务中使用。