Researches have demonstrated that low bit-width (e.g., INT8) quantization can
be employed to accelerate the inference process. It makes the gradient
quantization very promising since the backward propagation requires
approximately twice more computation than forward one. Due to the variability
and uncertainty of gradient distribution, a lot of methods have been proposed
to attain training stability. However, most of them ignore the channel-wise
gradient distributions and the impact of gradients with different magnitudes,
resulting in the degradation of final accuracy. In this paper, we propose a
novel INT8 quantization training framework for convolutional neural network to
address the above issues. Specifically, we adopt Gradient Vectorized
Quantization to quantize the gradient, based on the observation that layer-wise
gradients contain multiple distributions along the channel dimension. Then,
Magnitude-aware Clipping Strategy is introduced by taking the magnitudes of
gradients into consideration when minimizing the quantization error, and we
present a theoretical derivation to solve the quantization parameters of
different distributions. Experimental results on broad range of computer vision
tasks, such as image classification, object detection and video classification,
demonstrate that the proposed Distribution Adaptive INT8 Quantization training
method has achieved almost lossless training accuracy for different backbones,
including ResNet, MobileNetV2, InceptionV3, VGG and AlexNet, which is superior
to the state-of-the-art techniques. Moreover, we further implement the INT8
kernel that can accelerate the training iteration more than 200% under the
latest Turing architecture, i.e., our method excels on both training accuracy
and speed.

本文提出了一种新的 INT8 量化训练框架，其中采用梯度矢量化量化梯度，并引入基于梯度幅度的裁剪策略来解决量化梯度的误差，实验结果表明，在图像分类、物体检测和视频分类等广泛的计算机视觉任务中，所提出的 Distribution Adaptive INT8 Quantization 训练方法已经达到了几乎无损的训练精度，优于现有技术，并且这种方法能够在最新的 Turing 架构下实现训练速度提高 200% 以上。