Sharpness-Aware Minimization (SAM) is most known for achieving state-of the-art performances on natural image and language tasks. However, its most pronounced improvements (of tens of percent) is rather in the presence of label noise. Understanding SAM's label noise robustness requires a departure from characterizing the robustness of minimas lying in "flatter" regions of the loss landscape. In particular, the peak performance under label noise occurs with early stopping, far before the loss converges. We decompose SAM's robustness into two effects: one induced by changes to the logit term and the other induced by changes to the network Jacobian. The first can be observed in linear logistic regression where SAM provably up-weights the gradient contribution from clean examples. Although this explicit up-weighting is also observable in neural networks, when we intervene and modify SAM to remove this effect, surprisingly, we see no visible degradation in performance. We infer that SAM's effect in deeper networks is instead explained entirely by the effect SAM has on the network Jacobian. We theoretically derive the implicit regularization induced by this Jacobian effect in two layer linear networks. Motivated by our analysis, we see that cheaper alternatives to SAM that explicitly induce these regularization effects largely recover the benefits in deep networks trained on real-world datasets.

Sharpness-Aware Minimization对标签噪声鲁棒性的理解需要考虑到对误差曲面中“较平坦”区域最小值的鲁棒性以及提前停止对峰值性能的影响。通过对logit项和网络Jacobian的改变进行分解，我们推断深层网络中的SAM效果完全由其对网络Jacobian的影响解释，进一步推导了这种Jacobian效果在两层线性网络中引发的隐式正则化效应。受我们分析的启发，我们发现在真实数据集上训练的深度网络中，明确引入这些正则化效应的成本更低的SAM替代方法在很大程度上恢复了优势。

SAM对标签噪声具有鲁棒性的原因