When tasks change over time, meta-transfer learning seeks to improve the
efficiency of learning a new task via both meta-learning and transfer-learning.
While the standard attention has been effective in a variety of settings, we
question its effectiveness in improving meta-transfer learning since the tasks
being learned are dynamic and the amount of context can be substantially
smaller. In this paper, using a recently proposed meta-transfer learning model,
Sequential Neural Processes (SNP), we first empirically show that it suffers
from a similar underfitting problem observed in the functions inferred by
Neural Processes. However, we further demonstrate that unlike the meta-learning
setting, the standard attention mechanisms are not effective in meta-transfer
setting. To resolve, we propose a new attention mechanism, Recurrent Memory
Reconstruction (RMR), and demonstrate that providing an imaginary context that
is recurrently updated and reconstructed with interaction is crucial in
achieving effective attention for meta-transfer learning. Furthermore,
incorporating RMR into SNP, we propose Attentive Sequential Neural
Processes-RMR (ASNP-RMR) and demonstrate in various tasks that ASNP-RMR
significantly outperforms the baselines.

本研究介绍了一种新的抽象注意机制 Recurrent Memory Reconstruction (RMR)，并采用其改进了基于 Sequential Neural Processes (SNP) 的 meta-transfer learning 模型，所得到的 Attentive Sequential Neural Processes-RMR (ASNP-RMR) 在各项评估中表现显著优于基线模型。