Quantization has emerged as an essential technique for deploying deep neural
networks (DNNs) on devices with limited resources. However, quantized models
exhibit vulnerabilities when exposed to various noises in real-world
applications. Despite the importance of evaluating the impact of quantization
on robustness, existing research on this topic is limited and often disregards
established principles of robustness evaluation, resulting in incomplete and
inconclusive findings. To address this gap, we thoroughly evaluated the
robustness of quantized models against various noises (adversarial attacks,
natural corruptions, and systematic noises) on ImageNet. The comprehensive
evaluation results empirically provide valuable insights into the robustness of
quantized models in various scenarios, for example: (1) quantized models
exhibit higher adversarial robustness than their floating-point counterparts,
but are more vulnerable to natural corruptions and systematic noises; (2) in
general, increasing the quantization bit-width results in a decrease in
adversarial robustness, an increase in natural robustness, and an increase in
systematic robustness; (3) among corruption methods, \textit{impulse noise} and
\textit{glass blur} are the most harmful to quantized models, while
\textit{brightness} has the least impact; (4) among systematic noises, the
\textit{nearest neighbor interpolation} has the highest impact, while bilinear
interpolation, cubic interpolation, and area interpolation are the three least
harmful. Our research contributes to advancing the robust quantization of
models and their deployment in real-world scenarios.

通过对 ImageNet 上的各种噪声（对抗性攻击、自然扰动和系统噪声）进行了全面评估，我们得出了关于量化模型在各种场景下的鲁棒性的宝贵见解：（1）量化模型在对抗性鲁棒性方面表现更好，但对自然扰动和系统噪声更加脆弱；（2）一般来说，增加量化位宽会导致对抗性鲁棒性降低，自然鲁棒性增强，系统鲁棒性增加；（3）在扰动方法中，由	extit {脉冲噪声} 和	extit {玻璃模糊} 对量化模型的影响最大，而	extit {亮度} 的影响最小；（4）在系统噪声中，	extit {最近邻插值} 的影响最大，而双线性插值、立方插值和区域插值的影响最小。我们的研究推动了模型的鲁棒量化及其在实际场景中的部署。

RobustMQ：量化模型鲁棒性性能评估

RobustMQ: Benchmarking Robustness of Quantized Models

Visual AI systems are vulnerable to natural and synthetic physical corruption
in the real-world. Such corruption often arises unexpectedly and alters the
model's performance. In recent years, the primary focus has been on adversarial
attacks. However, natural corruptions (e.g., snow, fog, dust) are an
omnipresent threat to visual AI systems and should be considered equally
important. Many existing works propose interesting solutions to train robust
models against natural corruption. These works either leverage image
augmentations, which come with the additional cost of model training, or place
suspicious patches in the scene to design unadversarial examples. In this work,
we propose the idea of naturalistic support artifacts (NSA) for robust
prediction. The NSAs are shown to be beneficial in scenarios where model
parameters are inaccessible and adding artifacts in the scene is feasible. The
NSAs are natural looking objects generated through artifact training using
DC-GAN to have high visual fidelity in the scene. We test against natural
corruptions on the Imagenette dataset and observe the improvement in prediction
confidence score by four times. We also demonstrate NSA's capability to
increase adversarial accuracy by 8\% on average. Lastly, we qualitatively
analyze NSAs using saliency maps to understand how they help improve prediction
confidence.

我们提出了自然支持工件（NSA）的概念，这些工件通过使用 DC-GAN 进行工件培训生成，具有高视觉保真度，在场景中增加这些工件有助于提高预测置信度。我们在 Imagenette 数据集上进行了对自然污染的测试，并观察到预测置信度得分提高了四倍。我们还展示了 NSA 提高 8％的对抗准确率的能力。最后，我们利用显著性图 qualitatively 分析了 NSA，以了解它们如何帮助提高预测置信度。

NSA: 自然支持工件提升网络可信度

NSA: Naturalistic Support Artifact to Boost Network Confidence

Quantization has emerged as an essential technique for deploying deep neural
networks (DNNs) on devices with limited resources. However, quantized models
exhibit vulnerabilities when exposed to various noises in real-world
applications. Despite the importance of evaluating the impact of quantization
on robustness, existing research on this topic is limited and often disregards
established principles of robustness evaluation, resulting in incomplete and
inconclusive findings. To address this gap, we thoroughly evaluated the
robustness of quantized models against various noises (adversarial attacks,
natural corruptions, and systematic noises) on ImageNet. Extensive experiments
demonstrate that lower-bit quantization is more resilient to adversarial
attacks but is more susceptible to natural corruptions and systematic noises.
Notably, our investigation reveals that impulse noise (in natural corruptions)
and the nearest neighbor interpolation (in systematic noises) have the most
significant impact on quantized models. Our research contributes to advancing
the robust quantization of models and their deployment in real-world scenarios.

通过对 ImageNet 上的广泛实验，我们发现量化技术在部署深度神经网络时，虽然对于对抗性攻击具有更高的稳健性，但更容易受到自然噪声和系统噪声等噪声的影响。特别的，脉冲噪声和最邻近插值对量化模型影响最大，我们的研究有助于进一步提升模型的稳健性，使其能够在真实场景中部署应用。