Neural Processes (NPs) are efficient methods for estimating predictive
uncertainties. NPs comprise of a conditioning phase where a context dataset is
encoded, a querying phase where the model makes predictions using the context
dataset encoding, and an updating phase where the model updates its encoding
with newly received datapoints. However, state-of-the-art methods require
additional memory which scales linearly or quadratically with the size of the
dataset, limiting their applications, particularly in low-resource settings. In
this work, we propose Constant Memory Attentive Neural Processes (CMANPs), an
NP variant which only requires constant memory for the conditioning, querying,
and updating phases. In building CMANPs, we propose Constant Memory Attention
Block (CMAB), a novel general-purpose attention block that can compute its
output in constant memory and perform updates in constant computation.
Empirically, we show CMANPs achieve state-of-the-art results on meta-regression
and image completion tasks while being (1) significantly more memory efficient
than prior methods and (2) more scalable to harder settings.

本研究提出了一个新的神经过程模型 —— 常数内存注意力神经过程模型（CMANPs），它可以使用常数内存完成条件、查询和更新阶段，能够在低资源环境下高效地进行元回归和图像完成任务，与现有方法相比具有更好的内存效率和可扩展性。