It is critical to deploy complicated neural network models on hardware with
limited resources. This paper proposes a novel model quantization method, named
the Low-Cost Proxy-Based Adaptive Mixed-Precision Model Quantization (LCPAQ),
which contains three key modules. The hardware-aware module is designed by
considering the hardware limitations, while an adaptive mixed-precision
quantization module is developed to evaluate the quantization sensitivity by
using the Hessian matrix and Pareto frontier techniques. Integer linear
programming is used to fine-tune the quantization across different layers. Then
the low-cost proxy neural architecture search module efficiently explores the
ideal quantization hyperparameters. Experiments on the ImageNet demonstrate
that the proposed LCPAQ achieves comparable or superior quantization accuracy
to existing mixed-precision models. Notably, LCPAQ achieves 1/200 of the search
time compared with existing methods, which provides a shortcut in practical
quantization use for resource-limited devices.

该论文提出了一种名为 LCPAQ 的新型模型量化方法，其中包含三个关键模块。在考虑硬件限制的基础上设计了硬件感知模块，开发了自适应混合精度量化模块来评估量化灵敏度，并使用希狄矩阵和 Pareto 边界技术进行整数线性规划来微调不同层的量化。然后，低成本代理神经架构搜索模块高效探索理想的量化超参数。在 ImageNet 上的实验证明，提出的 LCPAQ 在量化精度方面达到了与现有混合精度模型相当甚至更好的水平。值得注意的是，与现有方法相比，LCPAQ 的搜索时间减少了 1/200，从而为资源有限设备的实际量化使用提供了捷径。