Energy-efficient deep neural network (DNN) accelerators are prone to non-idealities that degrade DNN performance at inference time. To mitigate such degradation, existing methods typically add perturbations to the DNN weights during training to simulate inference on noisy hardware. However, this often requires knowledge about the target hardware and leads to a trade-off between DNN performance and robustness, decreasing the former to increase the latter. In this work, we show that applying sharpness-aware training by optimizing for both the loss value and the loss sharpness significantly improves robustness to noisy hardware at inference time while also increasing DNN performance. We further motivate our results by showing a high correlation between loss sharpness and model robustness. We show superior performance compared to injecting noise during training and aggressive weight clipping on multiple architectures, optimizers, datasets, and training regimes without relying on any assumptions about the target hardware. This is observed on a generic noise model as well as on accurate noise simulations from real hardware.

本文提出了一种自适应尖锐度感知（SAMSON）的方法，通过优化损失值和损失尖锐度来提高神经网络模型对噪音硬件的稳健性，相比现有的尖锐度感知训练方法在无噪声和有噪声的环境下均表现出更好的模型泛化性能和鲁棒性。

SAMSON: 适应性异常值标准化缩放的尖锐度感知最小化，用于提高DNN的泛化性和鲁棒性