In order to deploy deep models in a computationally efficient manner, model
quantization approaches have been frequently used. In addition, as new hardware
that supports mixed bitwidth arithmetic operations, recent research on mixed
precision quantization (MPQ) begins to fully leverage the capacity of
representation by searching optimized bitwidths for different layers and
modules in a network. However, previous studies mainly search the MPQ strategy
in a costly scheme using reinforcement learning, neural architecture search,
etc., or simply utilize partial prior knowledge for bitwidth assignment, which
might be biased and sub-optimal. In this work, we present a novel Stochastic
Differentiable Quantization (SDQ) method that can automatically learn the MPQ
strategy in a more flexible and globally-optimized space with smoother gradient
approximation. Particularly, Differentiable Bitwidth Parameters (DBPs) are
employed as the probability factors in stochastic quantization between adjacent
bitwidth choices. After the optimal MPQ strategy is acquired, we further train
our network with entropy-aware bin regularization and knowledge distillation.
We extensively evaluate our method for several networks on different hardware
(GPUs and FPGA) and datasets. SDQ outperforms all state-of-the-art mixed or
single precision quantization with a lower bitwidth and is even better than the
full-precision counterparts across various ResNet and MobileNet families,
demonstrating the effectiveness and superiority of our method.

本文提出了一种新的随机可微量化（SDQ）方法，利用不同层和模块的优化比特宽度来自动学习混合精度量化策略，并利用熵感知的分 bin 正则化和知识蒸馏对网络进行训练，经过在不同硬件和数据集上广泛的评估，SDQ 在较低的比特宽度下优于所有最先进的混合或单精度量化，甚至优于各种 ResNet 和 MobileNet 家族的全精度对应物，展示了我们的方法的有效性和优越性。