Neural processes (NPs) are a powerful family of meta-learning models that
seek to approximate the posterior predictive map of the ground-truth stochastic
process from which each dataset in a meta-dataset is sampled. There are many
cases in which practitioners, besides having access to the dataset of interest,
may also have access to other datasets that share similarities with it. In this
case, integrating these datasets into the NP can improve predictions. We equip
NPs with this functionality and describe this paradigm as in-context in-context
learning. Standard NP architectures, such as the convolutional conditional NP
(ConvCNP) or the family of transformer neural processes (TNPs), are not capable
of in-context in-context learning, as they are only able to condition on a
single dataset. We address this shortcoming by developing the in-context
in-context learning pseudo-token TNP (ICICL-TNP). The ICICL-TNP builds on the
family of PT-TNPs, which utilise pseudo-token-based transformer architectures
to sidestep the quadratic computational complexity associated with regular
transformer architectures. Importantly, the ICICL-TNP is capable of
conditioning on both sets of datapoints and sets of datasets, enabling it to
perform in-context in-context learning. We demonstrate the importance of
in-context in-context learning and the effectiveness of the ICICL-TNP in a
number of experiments.

神经过程（NPs）是一类强大的元学习模型，旨在逼近元数据集中每个数据集从中抽样得到的地面真实随机过程的后验预测映射。我们在 NP 中增加了集成其他类似数据集的功能，描述了此范例作为上下文中的上下文学习。标准的 NP 架构（如卷积条件 NP（ConvCNP）或转换器神经过程（TNPs）系列）无法进行上下文中的上下文学习，因为它们只能在单个数据集上进行条件。我们通过开发上下文中的伪标记 TNP（ICICL-TNP）来解决这个问题。 ICICL-TNP 基于 PT-TNPs 系列，利用基于伪标记的转换器架构来规避常规转换器架构的二次计算复杂性。重要的是，ICICL-TNP 能够在数据点集和数据集集上进行条件修正，从而实现上下文中的上下文学习。我们在多个实验中证明了上下文中的上下文学习的重要性和 ICICL-TNP 的有效性。