Model Agnostic Meta-Learning (MAML) has emerged as a standard framework for meta-learning, where a meta-model is learned with the ability of fast adapting to new tasks. However, as a double-looped optimization problem, MAML needs to differentiate through the whole inner-loop optimization path for every outer-loop training step, which may lead to both computational inefficiency and sub-optimal solutions. In this paper, we generalize MAML to allow meta-learning to be defined in function spaces, and propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK). Within this paradigm, we introduce two meta-learning algorithms in the RKHS, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework. We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory. Extensive experimental studies demonstrate advantages of our paradigm in both efficiency and quality of solutions compared to related meta-learning algorithms. Another interesting feature of our proposed methods is that they are demonstrated to be more robust to adversarial attacks and out-of-distribution adaptation than popular baselines, as demonstrated in our experiments.

本研究将MAML泛化为能够被定义在函数空间内的元学习范式，并提出了在元模型的神经正切核引发的再生核希尔伯特空间（RKHS）中的首个元学习算法，该方法不再需要 MAML 框架中的子优化内循环适应，而是通过在RKHS中替换适应性并基于NTK理论解析解决适应性。大量实验表明，与相关的元学习算法相比，我们的模型在解决方案的效率和质量方面具有优势，并且比流行基线更具鲁棒性，可以更好地对抗对抗性攻击和分布适应性，这是我们提出方法的另一个有趣特点。

神经切向核元学习