Mixed-precision quantization can potentially achieve the optimal tradeoff
between performance and compression rate of deep neural networks, and thus,
have been widely investigated. However, it lacks a systematic method to
determine the exact quantization scheme. Previous methods either examine only a
small manually-designed search space or utilize a cumbersome neural
architecture search to explore the vast search space. These approaches cannot
lead to an optimal quantization scheme efficiently. This work proposes
bit-level sparsity quantization (BSQ) to tackle the mixed-precision
quantization from a new angle of inducing bit-level sparsity. We consider each
bit of quantized weights as an independent trainable variable and introduce a
differentiable bit-sparsity regularizer. BSQ can induce all-zero bits across a
group of weight elements and realize the dynamic precision reduction, leading
to a mixed-precision quantization scheme of the original model. Our method
enables the exploration of the full mixed-precision space with a single
gradient-based optimization process, with only one hyperparameter to tradeoff
the performance and compression. BSQ achieves both higher accuracy and higher
bit reduction on various model architectures on the CIFAR-10 and ImageNet
datasets comparing to previous methods.

提出了比特级稀疏量化法（BSQ），通过引入可微比特稀疏正则化，考虑量化权重的每一位作为一个独立可训练变量，实现了每组权重元素的全零位诱导以及实现动态精度降低，使得在 CIFAR-10 和 ImageNet 数据集上相对于以往方法，BSQ 在各种模型体系结构上获得更高的准确性和比特削减。