BriefGPT.xyz
May, 2023
正则化在尖锐度感知极小化中的关键作用
The Crucial Role of Normalization in Sharpness-Aware Minimization
HTML
PDF
Yan Dai, Kwangjun Ahn, Suvrit Sra
TL;DR
本研究探讨了 Sharpness-Aware Minimization(SAM) 优化算法中 normalization 组件对于优化效果的影响，发现它在稳定算法和漂移的最小值连续谱上都发挥着重要作用，这使得SAM算法具有很好的鲁棒性。
Abstract
sharpness-aware minimization
(SAM) is a recently proposed
gradient-based optimizer
(Foret et al., ICLR 2021) that greatly improves the prediction performance of
→