Model-agnostic meta-learning (MAML) is arguably one of the most popular
meta-learning algorithms nowadays. Nevertheless, its performance on few-shot
classification is far behind many recent algorithms dedicated to the problem.
In this paper, we point out several key facets of how to train MAML to excel in
few-shot classification. First, we find that MAML needs a large number of
gradient steps in its inner loop update, which contradicts its common usage in
few-shot classification. Second, we find that MAML is sensitive to the class
label assignments during meta-testing. Concretely, MAML meta-trains the
initialization of an $N$-way classifier. These $N$ ways, during meta-testing,
then have "$N!$" different permutations to be paired with a few-shot task of
$N$ novel classes. We find that these permutations lead to a huge variance of
accuracy, making MAML unstable in few-shot classification. Third, we
investigate several approaches to make MAML permutation-invariant, among which
meta-training a single vector to initialize all the $N$ weight vectors in the
classification head performs the best. On benchmark datasets like MiniImageNet
and TieredImageNet, our approach, which we name UNICORN-MAML, performs on a par
with or even outperforms many recent few-shot classification algorithms,
without sacrificing MAML's simplicity.

在 Few-shot classification 这个问题上，我们探究了如何训练 MAML 表现得更美好，发现 MAML 需要大量的梯度步骤才能适应少样本分类，敏感于试验时分类标签的赋值，本文提供了多种方法解决其排列不变性。我们命名为 UNICORN-MAML 的方法在 MiniImageNet 和 TieriImageNet 等基准数据集上表现良好，并与许多最新的少样本分类算法不相上下，而不损失 MAML 的简单性。

如何训练你的 MAML 在少样本分类领域表现卓越

How to Train Your MAML to Excel in Few-Shot Classification

We introduce the continuized Nesterov acceleration, a close variant of
Nesterov acceleration whose variables are indexed by a continuous time
parameter. The two variables continuously mix following a linear ordinary
differential equation and take gradient steps at random times. This continuized
variant benefits from the best of the continuous and the discrete frameworks:
as a continuous process, one can use differential calculus to analyze
convergence and obtain analytical expressions for the parameters; and a
discretization of the continuized process can be computed exactly with
convergence rates similar to those of Nesterov original acceleration. We show
that the discretization has the same structure as Nesterov acceleration, but
with random parameters. We provide continuized Nesterov acceleration under
deterministic as well as stochastic gradients, with either additive or
multiplicative noise. Finally, using our continuized framework and expressing
the gossip averaging problem as the stochastic minimization of a certain energy
function, we provide the first rigorous acceleration of asynchronous gossip
algorithms.

本文提出了连续 Nesterov 加速法，将 Nesterov 加速法的变量用连续时间参数索引，使两个变量连续混合，其间隔时间内随机进行梯度步骤。我们证明了该变体具有与 Nesterov 原始加速法相似的收敛率，并且具有连续和离散框架的最佳性能。我们展示了连续 Nesterov 加速法在随机 / 确定梯度及其噪声下的应用，并将异步 gossip 算法的问题表示为某种能量函数的随机最小化问题，提供了第一个基于该连续框架的异步 gossip 加速定理。

随机梯度下降和随机八卦算法中的 Nesterov 加速算法连续化视角

A Continuized View on Nesterov Acceleration for Stochastic Gradient  Descent and Randomized Gossip

This paper considers unconstrained convex optimization problems with
time-varying objective functions. We propose algorithms with a discrete
time-sampling scheme to find and track the solution trajectory based on
prediction and correction steps, while sampling the problem data at a constant
rate of $1/h$, where $h$ is the length of the sampling interval. The prediction
step is derived by analyzing the iso-residual dynamics of the optimality
conditions. The correction step adjusts for the distance between the current
prediction and the optimizer at each time step, and consists either of one or
multiple gradient steps or Newton steps, which respectively correspond to the
gradient trajectory tracking (GTT) or Newton trajectory tracking (NTT)
algorithms. Under suitable conditions, we establish that the asymptotic error
incurred by both proposed methods behaves as $O(h^2)$, and in some cases as
$O(h^4)$, which outperforms the state-of-the-art error bound of $O(h)$ for
correction-only methods in the gradient-correction step. Moreover, when the
characteristics of the objective function variation are not available, we
propose approximate gradient and Newton tracking algorithms (AGT and ANT,
respectively) that still attain these asymptotical error bounds. Numerical
simulations demonstrate the practical utility of the proposed methods and that
they improve upon existing techniques by several orders of magnitude.

本文提出一种基于预测和修正步骤的算法，采用离散时间采样方案，在每个时间步长采样一次问题数据，以找到和跟踪解轨迹，用于无约束凸优化问题，并且在某些情况下，渐近误差仅为 O（h ^ 2），大大优于梯度校正步骤仅误差为 O（h）的现有技术。