Network quantization aims at reducing bit-widths of weights and/or
activations, particularly important for implementing deep neural networks with
limited hardware resources. Most methods use the straight-through estimator
(STE) to train quantized networks, which avoids a zero-gradient problem by
replacing a derivative of a discretizer (i.e., a round function) with that of
an identity function. Although quantized networks exploiting the STE have shown
decent performance, the STE is sub-optimal in that it simply propagates the
same gradient without considering discretization errors between inputs and
outputs of the discretizer. In this paper, we propose an element-wise gradient
scaling (EWGS), a simple yet effective alternative to the STE, training a
quantized network better than the STE in terms of stability and accuracy. Given
a gradient of the discretizer output, EWGS adaptively scales up or down each
gradient element, and uses the scaled gradient as the one for the discretizer
input to train quantized networks via backpropagation. The scaling is performed
depending on both the sign of each gradient element and an error between the
continuous input and discrete output of the discretizer. We adjust a scaling
factor adaptively using Hessian information of a network. We show extensive
experimental results on the image classification datasets, including CIFAR-10
and ImageNet, with diverse network architectures under a wide range of
bit-width settings, demonstrating the effectiveness of our method.

本文提出了一种新的基于 element-wise gradient scaling 的量化网络的训练方法，该方法可以更好地解决量化误差问题，提高在有限硬件资源情况下深度神经网络的稳定性和准确性。

使用逐元素梯度缩放的网络量化

Network Quantization with Element-wise Gradient Scaling

Recent machine learning methods use increasingly large deep neural networks
to achieve state of the art results in various tasks. The gains in performance
come at the cost of a substantial increase in computation and storage
requirements. This makes real-time implementations on limited resources
hardware a challenging task. One popular approach to address this challenge is
to perform low-bit precision computations via neural network quantization.
However, aggressive quantization generally entails a severe penalty in terms of
accuracy, and often requires retraining of the network, or resorting to higher
bit precision quantization. In this paper, we formalize the linear quantization
task as a Minimum Mean Squared Error (MMSE) problem for both weights and
activations, allowing low-bit precision inference without the need for full
network retraining. The main contributions of our approach are the
optimizations of the constrained MSE problem at each layer of the network, the
hardware aware partitioning of the network parameters, and the use of multiple
low precision quantized tensors for poorly approximated layers. The proposed
approach allows 4 bits integer (INT4) quantization for deployment of pretrained
models on limited hardware resources. Multiple experiments on various network
architectures show that the suggested method yields state of the art results
with minimal loss of tasks accuracy.

本文提出了一种在有限硬件资源上实现预训练模型的 4 位整数（INT4）量化的优化方法，将线性量化任务形式化为最小均方误差（MMSE）问题，并对网络的每一层进行限制 MSE 问题的优化以及多个量化张量的硬件感知分区，除少量精度降低外，在多种网络架构上实现最先进的结果。