In this work we study generalization of neural networks in gradient-based meta-learning by analyzing various properties of the objective landscapes. We experimentally demonstrate that as meta-training progresses, the meta-test solutions, obtained after adapting the meta-train solution of the model, to new tasks via few steps of gradient-based fine-tuning, become flatter, lower in loss, and further away from the meta-train solution. We also show that those meta-test solutions become flatter even as generalization starts to degrade, thus providing an experimental evidence against the correlation between generalization and flat minima in the paradigm of gradient-based meta-leaning. Furthermore, we provide empirical evidence that generalization to new tasks is correlated with the coherence between their adaptation trajectories in parameter space, measured by the average cosine similarity between task-specific trajectory directions, starting from a same meta-train solution. We also show that coherence of meta-test gradients, measured by the average inner product between the task-specific gradient vectors evaluated at meta-train solution, is also correlated with generalization. Based on these observations, we propose a novel regularizer for MAML and provide experimental evidence for its effectiveness.

本文研究了神经网络在基于梯度的元学习中的泛化问题，分析了目标景观的各种特性，提出了一种新的正则化方法以增强模型的泛化能力。实验表明，在元训练提供的元解决方案的基础上，通过几步基于梯度的微调适应元训练模型到新任务时，所得到的元测试解决方案变得越来越平坦，损失更低，并且远离元训练解决方案。

基于梯度元学习的泛化理解探索