Improving the hardware efficiency of deep neural network (DNN) accelerators
with techniques such as quantization and sparsity enhancement have shown an
immense promise. However, their inference accuracy in non-ideal real-world
settings (such as in the presence of hardware faults) is yet to be
systematically analyzed. In this work, we investigate the impact of memory
faults on activation-sparse quantized DNNs (AS QDNNs). We show that a high
level of activation sparsity comes at the cost of larger vulnerability to
faults, with AS QDNNs exhibiting up to 11.13% lower accuracy than the standard
QDNNs. We establish that the degraded accuracy correlates with a sharper minima
in the loss landscape for AS QDNNs, which makes them more sensitive to
perturbations in the weight values due to faults. Based on this observation, we
employ sharpness-aware quantization (SAQ) training to mitigate the impact of
memory faults. The AS and standard QDNNs trained with SAQ have up to 19.50% and
15.82% higher inference accuracy, respectively compared to their conventionally
trained equivalents. Moreover, we show that SAQ-trained AS QDNNs show higher
accuracy in faulty settings than standard QDNNs trained conventionally. Thus,
sharpness-aware training can be instrumental in achieving sparsity-related
latency benefits without compromising on fault tolerance.

提高深度神经网络（DNN）加速器的硬件效率，例如采用量化和稀疏增强技术，显示了巨大的潜力。然而，在非理想的现实环境（例如存在硬件故障）中，它们的推理准确性尚未被系统地分析。本文研究了存储器故障对激活稀疏量化 DNN（AS QDNN）的影响，发现激活稀疏度较高会增加故障的脆弱性，导致 AS QDNN 的准确性比标准 QDNN 低达 11.13%。基于这一观察，我们采用了锐度感知量化（SAQ）训练以缓解存储器故障的影响，结果表明使用 SAQ 训练的 AS QDNN 和标准 QDNN 相比于传统训练方法，推理准确性分别提高了 19.50% 和 15.82%。此外，我们还表明 SAQ 训练的 AS QDNN 在故障环境中比传统训练方法的标准 QDNN 具有更高的准确性。因此，锐度感知训练可以在实现稀疏相关的延迟优势的同时保持容错性。

激活稀疏量化深度神经网络中的存储错误：基于锐度感知训练的分析和缓解

Memory Faults in Activation-sparse Quantized Deep Neural Networks:  Analysis and Mitigation using Sharpness-aware Training

Quantization is a promising approach for reducing memory overhead and
accelerating inference, especially in large pre-trained language model (PLM)
scenarios. While having no access to original training data due to security and
privacy concerns has emerged the demand for zero-shot quantization. Most of the
cutting-edge zero-shot quantization methods primarily 1) apply to computer
vision tasks, and 2) neglect of overfitting problem in the generative
adversarial learning process, leading to sub-optimal performance. Motivated by
this, we propose a novel zero-shot sharpness-aware quantization (ZSAQ)
framework for the zero-shot quantization of various PLMs. The key algorithm in
solving ZSAQ is the SAM-SGA optimization, which aims to improve the
quantization accuracy and model generalization via optimizing a minimax
problem. We theoretically prove the convergence rate for the minimax
optimization problem and this result can be applied to other nonconvex-PL
minimax optimization frameworks. Extensive experiments on 11 tasks demonstrate
that our method brings consistent and significant performance gains on both
discriminative and generative PLMs, i.e., up to +6.98 average score.
Furthermore, we empirically validate that our method can effectively improve
the model generalization.

我们提出了一种新颖的零射频锐化感知量化（ZSAQ）框架，用于各种预训练语言模型的零射频量化，并理论证明了 SAM-SGA 优化算法的收敛速度以及该结果适用于其他非凸型极小 - 极大优化框架。通过在 11 个任务上进行大量实验证明，我们的方法在鉴别性和生成性预训练语言模型上都带来了一致且显著的性能提升，即最高可达 + 6.98 的平均得分，并且在实践中验证了我们的方法能够有效改善模型的泛化能力。

零炮灰锐度感知量化预训练语言模型

Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models

Network quantization is a dominant paradigm of model compression. However,
the abrupt changes in quantized weights during training often lead to severe
loss fluctuations and result in a sharp loss landscape, making the gradients
unstable and thus degrading the performance. Recently, Sharpness-Aware
Minimization (SAM) has been proposed to smooth the loss landscape and improve
the generalization performance of the models. Nevertheless, directly applying
SAM to the quantized models can lead to perturbation mismatch or diminishment
issues, resulting in suboptimal performance. In this paper, we propose a novel
method, dubbed Sharpness-Aware Quantization (SAQ), to explore the effect of SAM
in model compression, particularly quantization for the first time.
Specifically, we first provide a unified view of quantization and SAM by
treating them as introducing quantization noises and adversarial perturbations
to the model weights, respectively. According to whether the noise and
perturbation terms depend on each other, SAQ can be formulated into three
cases, which are analyzed and compared comprehensively. Furthermore, by
introducing an efficient training strategy, SAQ only incurs a little additional
training overhead compared with the default optimizer (e.g., SGD or AdamW).
Extensive experiments on both convolutional neural networks and Transformers
across various datasets (i.e., ImageNet, CIFAR-10/100, Oxford Flowers-102,
Oxford-IIIT Pets) show that SAQ improves the generalization performance of the
quantized models, yielding the SOTA results in uniform quantization. For
example, on ImageNet, SAQ outperforms AdamW by 1.2% on the Top-1 accuracy for
4-bit ViT-B/16. Our 4-bit ResNet-50 surpasses the previous SOTA method by 0.9%
on the Top-1 accuracy.

本研究提出了一种新方法，叫做 sharpness-aware quantization，旨在通过减小量化时的抖动来提高网络压缩的泛化性能，此方法在多组实验中均能取得比当前最先进方法更好的结果。