Although model-agnostic meta-learning (MAML) is a very successful algorithm
in meta-learning practice, it can have high computational cost because it
updates all model parameters over both the inner loop of task-specific
adaptation and the outer-loop of meta initialization training. A more efficient
algorithm ANIL (which refers to almost no inner loop) was proposed recently by
Raghu et al. 2019, which adapts only a small subset of parameters in the inner
loop and thus has substantially less computational cost than MAML as
demonstrated by extensive experiments. However, the theoretical convergence of
ANIL has not been studied yet. In this paper, we characterize the convergence
rate and the computational complexity for ANIL under two representative
inner-loop loss geometries, i.e., strongly-convexity and nonconvexity. Our
results show that such a geometric property can significantly affect the
overall convergence performance of ANIL. For example, ANIL achieves a faster
convergence rate for a strongly-convex inner-loop loss as the number $N$ of
inner-loop gradient descent steps increases, but a slower convergence rate for
a nonconvex inner-loop loss as $N$ increases. Moreover, our complexity analysis
provides a theoretical quantification on the improved efficiency of ANIL over
MAML. The experiments on standard few-shot meta-learning benchmarks validate
our theoretical findings.

该论文研究了基于内部循环损失的几何性质如何影响 ANIL 算法的收敛速度和计算复杂性。研究表明，ANIL 对于强凸内环损失的收敛速度随着内环梯度下降步数的增加而加快，而对于非凸内环损失则会随着 N 的增加而减慢。并且，对于标准的几个元学习基准，我们进行了实验以验证我们的理论发现。