Robust quantization improves the tolerance of networks for various implementations, allowing reliable output in different bit-widths or fragmented low-precision arithmetic. In this work, we perform extensive analyses to identify the sources of quantization error and present three insights to robustify a network against quantization: reduction of error propagation, range clamping for error minimization, and inherited robustness against quantization. Based on these insights, we propose two novel methods called symmetry regularization (SymReg) and saturating nonlinearity (SatNL). Applying the proposed methods during training can enhance the robustness of arbitrary neural networks against quantization on existing post-training quantization (PTQ) and quantization-aware training (QAT) algorithms and enables us to obtain a single weight flexible enough to maintain the output quality under various conditions. We conduct extensive studies on CIFAR and ImageNet datasets and validate the effectiveness of the proposed methods.

本研究通过分析后，提出了“减少误差传播”、“范围固定以最小化误差”和“对抗量化的继承鲁棒性”等三个指导。结合这三个指导，我们提出了对称正则化和饱和非线性两种新方法，能够增强任意神经网络对量化的鲁棒性。实验证明这些方法在CIFAR和ImageNet数据集上有效。

对偶正则化和饱和非线性用于稳健量化