Recent advancements in learning algorithms have demonstrated that the
sharpness of the loss surface is an effective measure for improving the
generalization gap. Building upon this concept, Sharpness-Aware Minimization
(SAM) was proposed to enhance model generalization and achieved
state-of-the-art performance. SAM consists of two main steps, the weight
perturbation step and the weight updating step. However, the perturbation in
SAM is determined by only the gradient of the training loss, or cross-entropy
loss. As the model approaches a stationary point, this gradient becomes small
and oscillates, leading to inconsistent perturbation directions and also has a
chance of diminishing the gradient. Our research introduces an innovative
approach to further enhancing model generalization. We propose the Adaptive
Adversarial Cross-Entropy (AACE) loss function to replace standard
cross-entropy loss for SAM's perturbation. AACE loss and its gradient uniquely
increase as the model nears convergence, ensuring consistent perturbation
direction and addressing the gradient diminishing issue. Additionally, a novel
perturbation-generating function utilizing AACE loss without normalization is
proposed, enhancing the model's exploratory capabilities in near-optimum
stages. Empirical testing confirms the effectiveness of AACE, with experiments
demonstrating improved performance in image classification tasks using Wide
ResNet and PyramidNet across various datasets. The reproduction code is
available online

近期学习算法的进展表明，损失曲面的陡峭程度是提高泛化差距的有效指标。本研究引入创新的方法进一步增强模型泛化能力，提出了自适应对抗交叉熵（AACE）损失函数替代 Sharpness-Aware Minimization（SAM）的扰动，同时提出了一种利用 AACE 损失生成扰动的新方法，实验证实了 AACE 的有效性，并通过在各种数据集上使用 Wide ResNet 和 PyramidNet 进行图像分类任务的实验，取得了改进的性能。