In this paper, we hypothesize that gradient-based meta-learning (GBML)
implicitly suppresses the Hessian along the optimization trajectory in the
inner loop. Based on this hypothesis, we introduce an algorithm called SHOT
(Suppressing the Hessian along the Optimization Trajectory) that minimizes the
distance between the parameters of the target and reference models to suppress
the Hessian in the inner loop. Despite dealing with high-order terms, SHOT does
not increase the computational complexity of the baseline model much. It is
agnostic to both the algorithm and architecture used in GBML, making it highly
versatile and applicable to any GBML baseline. To validate the effectiveness of
SHOT, we conduct empirical tests on standard few-shot learning tasks and
qualitatively analyze its dynamics. We confirm our hypothesis empirically and
demonstrate that SHOT outperforms the corresponding baseline. Code is available
at: this https URL

本文中，我们假设基于梯度的元学习在内循环中隐式地压抑了海森矩阵。基于这一假设，我们提出了一种名为 SHOT（抑制优化轨迹上的海森矩阵）的算法，通过最小化目标模型和参考模型参数之间的距离来抑制内循环中的海森矩阵。尽管处理了高阶项，SHOT 并不增加基线模型的计算复杂度太多。它对于 GBML 使用的算法和体系结构都是不可知的，因此非常通用且适用于任何 GBML 基线。为了验证 SHOT 的有效性，我们在标准的少样本学习任务上进行了实证测试，并对其动态进行了定性分析。我们通过实验证实了我们的假设，并展示了 SHOT 优于对应基线的结果。代码可以在此网址获得：this https URL