This paper presents incremental network quantization (INQ), a novel method,
targeting to efficiently convert any pre-trained full-precision convolutional
neural network (CNN) model into a low-precision version whose weights are
constrained to be either powers of two or zero. Unlike existing methods which
are struggled in noticeable accuracy loss, our INQ has the potential to resolve
this issue, as benefiting from two innovations. On one hand, we introduce three
interdependent operations, namely weight partition, group-wise quantization and
re-training. A well-proven measure is employed to divide the weights in each
layer of a pre-trained CNN model into two disjoint groups. The weights in the
first group are responsible to form a low-precision base, thus they are
quantized by a variable-length encoding method. The weights in the other group
are responsible to compensate for the accuracy loss from the quantization, thus
they are the ones to be re-trained. On the other hand, these three operations
are repeated on the latest re-trained group in an iterative manner until all
the weights are converted into low-precision ones, acting as an incremental
network quantization and accuracy enhancement procedure. Extensive experiments
on the ImageNet classification task using almost all known deep CNN
architectures including AlexNet, VGG-16, GoogleNet and ResNets well testify the
efficacy of the proposed method. Specifically, at 5-bit quantization, our
models have improved accuracy than the 32-bit floating-point references. Taking
ResNet-18 as an example, we further show that our quantized models with 4-bit,
3-bit and 2-bit ternary weights have improved or very similar accuracy against
its 32-bit floating-point baseline. Besides, impressive results with the
combination of network pruning and INQ are also reported. The code is available
at this https URL

介绍了一种增量网络量化方法，该方法可以高效地将任何已训练好的卷积神经网络模型转换为低精度版本，它的权重被限制为二的幂次或零，并成功解决了现有方法存在的精度丢失问题。