Sharpness-aware minimization (SAM) has been instrumental in improving deep
neural network training by minimizing both the training loss and the sharpness
of the loss landscape, leading the model into flatter minima that are
associated with better generalization properties. In another aspect,
Model-Agnostic Meta-Learning (MAML) is a framework designed to improve the
adaptability of models. MAML optimizes a set of meta-models that are
specifically tailored for quick adaptation to multiple tasks with minimal
fine-tuning steps and can generalize well with limited data. In this work, we
explore the connection between SAM and MAML, particularly in terms of enhancing
model generalization. We introduce Agnostic-SAM, a novel approach that combines
the principles of both SAM and MAML. Agnostic-SAM adapts the core idea of SAM
by optimizing the model towards wider local minima using training data, while
concurrently maintaining low loss values on validation data. By doing so, it
seeks flatter minima that are not only robust to small perturbations but also
less vulnerable to data distributional shift problems. Our experimental results
demonstrate that Agnostic-SAM significantly improves generalization over
baselines across a range of datasets and under challenging conditions such as
noisy labels and data limitation.

在这项工作中，我们探索了锐度感知极小化（SAM）和模型不可知元学习（MAML）之间的联系，特别是在增强模型泛化方面。我们引入了一种新方法 Agnostic-SAM，它结合了 SAM 和 MAML 的原则。Agnostic-SAM 通过优化模型朝向具有更广泛的局部最小值，使用训练数据来调整 SAM 的核心思想，并同时在验证数据上保持较低的损失值。通过这样做，它寻求不仅对小扰动具有鲁棒性，而且对数据分布转换问题更加稳健的最小值。我们的实验结果表明，Agnostic-SAM 在各种数据集和具有挑战性的条件下（如噪声标签和数据限制）显著改善了泛化能力。