The exponentially large discrete search space in mixed-precision quantization
(MPQ) makes it hard to determine the optimal bit-width for each layer. Previous
works usually resort to iterative search methods on the training set, which
consume hundreds or even thousands of GPU-hours. In this study, we reveal that
some unique learnable parameters in quantization, namely the scale factors in
the quantizer, can serve as importance indicators of a layer, reflecting the
contribution of that layer to the final accuracy at certain bit-widths. These
importance indicators naturally perceive the numerical transformation during
quantization-aware training, which can precisely provide quantization
sensitivity metrics of layers. However, a deep network always contains hundreds
of such indicators, and training them one by one would lead to an excessive
time cost. To overcome this issue, we propose a joint training scheme that can
obtain all indicators at once. It considerably speeds up the indicators
training process by parallelizing the original sequential training processes.
With these learned importance indicators, we formulate the MPQ search problem
as a one-time integer linear programming (ILP) problem. That avoids the
iterative search and significantly reduces search time without limiting the
bit-width search space. For example, MPQ search on ResNet18 with our indicators
takes only 0.06 s, which improves time efficiency exponentially compared to
iterative search methods. Also, extensive experiments show our approach can
achieve SOTA accuracy on ImageNet for far-ranging models with various
constraints (e.g., BitOps, compress rate). Code is available on
this https URL

该研究提出利用量化器中的可学习参数作为量化精度重要指标，通过一次整数线性规划来确定混合精度量化的最佳位宽以提高时间效率，并在多种模型中实现了 SOTA 精度。