We introduce a Power-of-Two low-bit post-training quantization(PTQ) method
for deep neural network that meets hardware requirements and does not call for
long-time retraining. Power-of-Two quantization can convert the multiplication
introduced by quantization and dequantization to bit-shift that is adopted by
many efficient accelerators. However, the Power-of-Two scale factors have fewer
candidate values, which leads to more rounding or clipping errors. We propose a
novel Power-of-Two PTQ framework, dubbed RAPQ, which dynamically adjusts the
Power-of-Two scales of the whole network instead of statically determining them
layer by layer. It can theoretically trade off the rounding error and clipping
error of the whole network. Meanwhile, the reconstruction method in RAPQ is
based on the BN information of every unit. Extensive experiments on ImageNet
prove the excellent performance of our proposed method. Without bells and
whistles, RAPQ can reach accuracy of 65% and 48% on ResNet-18 and MobileNetV2
respectively with weight INT2 activation INT4. We are the first to propose the
more constrained but hardware-friendly Power-of-Two quantization scheme for
low-bit PTQ specially and prove that it can achieve nearly the same accuracy as
SOTA PTQ method. The code was released.

提出一种名为 RAPQ 的 Power-of-Two 低位后训练量化方法，动态调整整个网络的二的幂次方比例因子，用 BN 信息进行重构，证明该方法可以实现近似于 SOTA 方法的高精度，代码已发布。

RAPQ: 针对二次幂低比特后训练量化提高精度

RAPQ: Rescuing Accuracy for Power-of-Two Low-bit Post-training  Quantization

Deep convolutional neural networks (CNNs) are powerful tools for a wide range
of vision tasks, but the enormous amount of memory and compute resources
required by CNNs pose a challenge in deploying them on constrained devices.
Existing compression techniques, while excelling at reducing model sizes,
struggle to be computationally friendly. In this paper, we attend to the
statistical properties of sparse CNNs and present focused quantization, a novel
quantization strategy based on power-of-two values, which exploits the weight
distributions after fine-grained pruning. The proposed method dynamically
discovers the most effective numerical representation for weights in layers
with varying sparsities, significantly reducing model sizes. Multiplications in
quantized CNNs are replaced with much cheaper bit-shift operations for
efficient inference. Coupled with lossless encoding, we built a compression
pipeline that provides CNNs with high compression ratios (CR), low computation
cost and minimal loss in accuracy. In ResNet-50, we achieved a 18.08x CR with
only 0.24% loss in top-5 accuracy, outperforming existing compression methods.
We fully compressed a ResNet-18 and found that it is not only higher in CR and
top-5 accuracy, but also more hardware efficient as it requires fewer logic
gates to implement when compared to other state-of-the-art quantization methods
assuming the same throughput.

本文提出一种基于二次幂值的、专注于稀疏 CNN 的量化策略，并结合无损编码构建一种高压缩比、计算成本低、准确率几乎无损的压缩管线，在 ResNet-50 网络上实现 18.08x CR 的压缩效果，同时也证明更硬件有效。