Exploring the expected quantizing scheme with suitable mixed-precision policy
is the key point to compress deep neural networks (DNNs) in high efficiency and
accuracy. This exploration implies heavy workloads for domain experts, and an
automatic compression method is needed. However, the huge search space of the
automatic method introduces plenty of computing budgets that make the automatic
process challenging to be applied in real scenarios. In this paper, we propose
an end-to-end framework named AutoQNN, for automatically quantizing different
layers utilizing different schemes and bitwidths without any human labor.
AutoQNN can seek desirable quantizing schemes and mixed-precision policies for
mainstream DNN models efficiently by involving three techniques: quantizing
scheme search (QSS), quantizing precision learning (QPL), and quantized
architecture generation (QAG). QSS introduces five quantizing schemes and
defines three new schemes as a candidate set for scheme search, and then uses
the differentiable neural architecture search (DNAS) algorithm to seek the
layer- or model-desired scheme from the set. QPL is the first method to learn
mixed-precision policies by reparameterizing the bitwidths of quantizing
schemes, to the best of our knowledge. QPL optimizes both classification loss
and precision loss of DNNs efficiently and obtains the relatively optimal
mixed-precision model within limited model size and memory footprint. QAG is
designed to convert arbitrary architectures into corresponding quantized ones
without manual intervention, to facilitate end-to-end neural network
quantization. We have implemented AutoQNN and integrated it into Keras.
Extensive experiments demonstrate that AutoQNN can consistently outperform
state-of-the-art quantization.

本文提出一种名为 AutoQNN 的端到端框架，通过引入量化方案搜索（QSS）、量化精度学习（QPL）和量化架构生成（QAG）三种技术，自动实现对不同深度神经网络（DNN）模型的不同层进行不同方案和不同精度的量化，无需人工干预，实现对深度神经网络的自动量化。实验表明，AutoQNN 相对于现有的量化算法在准确性和效率上都表现得更优秀。