Weight quantization for deep ConvNets has shown promising results for
applications such as image classification and semantic segmentation and is
especially important for applications where memory storage is limited. However,
when aiming for quantization without accuracy degradation, different tasks may
end up with different bitwidths. This creates complexity for software and
hardware support and the complexity accumulates when one considers
mixed-precision quantization, in which case each layer's weights use a
different bitwidth. Our key insight is that optimizing for the least bitwidth
subject to no accuracy degradation is not necessarily an optimal strategy. This
is because one cannot decide optimality between two bitwidths if one has a
smaller model size while the other has better accuracy. In this work, we take
the first step to understand if some weight bitwidth is better than others by
aligning all to the same model size using a width-multiplier. Under this
setting, somewhat surprisingly, we show that using a single bitwidth for the
whole network can achieve better accuracy compared to mixed-precision
quantization targeting zero accuracy degradation when both have the same model
size. In particular, our results suggest that when the number of channels
becomes a target hyperparameter, a single weight bitwidth throughout the
network shows superior results for model compression.

本论文通过对模型宽度乘数的使用，研究了深度卷积神经网络中权重量化的最佳位宽选择，发现使用单一位宽比混合精度量化对模型压缩更优。

一个比特宽度统治全球

One Weight Bitwidth to Rule Them All

Second-order information has proven to be very effective in determining the
redundancy of neural network weights and activations. Recent paper proposes to
use Hessian traces of weights and activations for mixed-precision quantization
and achieves state-of-the-art results. However, prior works only focus on
selecting bits for each layer while the redundancy of different channels within
a layer also differ a lot. This is mainly because the complexity of determining
bits for each channel is too high for original methods. Here, we introduce
Channel-wise Hessian Aware trace-Weighted Quantization (CW-HAWQ). CW-HAWQ uses
Hessian trace to determine the relative sensitivity order of different channels
of activations and weights. What's more, CW-HAWQ proposes to use deep
Reinforcement learning (DRL) Deep Deterministic Policy Gradient (DDPG)-based
agent to find the optimal ratios of different quantization bits and assign bits
to channels according to the Hessian trace order. The number of states in
CW-HAWQ is much smaller compared with traditional AutoML based mix-precision
methods since we only need to search ratios for the quantization bits. Compare
CW-HAWQ with state-of-the-art shows that we can achieve better results for
multiple networks.

通过使用通道数量更少的针对 Hessian 迹的量化方法，结合使用基于深度强化学习的代理寻找最佳量化位和通道分配的方法，可以获得更好的结果。

通道维度 Hessian 感知迹加权量化神经网络

Channel-wise Hessian Aware trace-Weighted Quantization of Neural  Networks

Quantization is a promising approach for reducing the inference time and
memory footprint of neural networks. However, most existing quantization
methods require access to the original training dataset for retraining during
quantization. This is often not possible for applications with sensitive or
proprietary data, e.g., due to privacy and security concerns. Existing
zero-shot quantization methods use different heuristics to address this, but
they result in poor performance, especially when quantizing to ultra-low
precision. Here, we propose ZeroQ , a novel zero-shot quantization framework to
address this. ZeroQ enables mixed-precision quantization without any access to
the training or validation data. This is achieved by optimizing for a Distilled
Dataset, which is engineered to match the statistics of batch normalization
across different layers of the network. ZeroQ supports both uniform and
mixed-precision quantization. For the latter, we introduce a novel Pareto
frontier based method to automatically determine the mixed-precision bit
setting for all layers, with no manual search involved. We extensively test our
proposed method on a diverse set of models, including ResNet18/50/152,
MobileNetV2, ShuffleNet, SqueezeNext, and InceptionV3 on ImageNet, as well as
RetinaNet-ResNet50 on the Microsoft COCO dataset. In particular, we show that
ZeroQ can achieve 1.71\% higher accuracy on MobileNetV2, as compared to the
recently proposed DFQ method. Importantly, ZeroQ has a very low computational
overhead, and it can finish the entire quantization process in less than 30s
(0.5\% of one epoch training time of ResNet50 on ImageNet). We have
open-sourced the ZeroQ
framework\footnote{this https URL}.

本文提出了一种名为 ZeroQ 的新型零 - shot 量化框架，用于量化神经网络而无需访问原始数据，ZeroQ 通过优化一个与网络不同层的批量归一化统计数据相匹配的 Distilled Dataset 来实现。我们在不同的模型上进行了广泛测试，证明 ZeroQ 可以实现比 DFQ 方法更高的精度，具有极低的计算开销。

ZeroQ：一种新颖的零样本量化框架

ZeroQ: A Novel Zero Shot Quantization Framework

Neural networks have shown great performance in cognitive tasks. When
deploying network models on mobile devices with limited resources, weight
quantization has been widely adopted. Binary quantization obtains the highest
compression but usually results in big accuracy drop. In practice, 8-bit or
16-bit quantization is often used aiming at maintaining the same accuracy as
the original 32-bit precision. We observe different layers have different
accuracy sensitivity of quantization. Thus judiciously selecting different
precision for different layers/structures can potentially produce more
efficient models compared to traditional quantization methods by striking a
better balance between accuracy and compression rate. In this work, we propose
a fine-grained quantization approach for deep neural network compression by
relaxing the search space of quantization bitwidth from discrete to a
continuous domain. The proposed approach applies gradient descend based
optimization to generate a mixed-precision quantization scheme that outperforms
the accuracy of traditional quantization methods under the same compression
rate.

本文提出了一种基于梯度下降优化的深度神经网络压缩的精细化量化方法，通过在不同结构、层次上采用不同的精度，达到更好的压缩比和准确率的平衡。实验结果表明，与传统量化方法相比，该方法在相同压缩率下表现更优。