Neural network quantization is a very promising solution in the field of
model compression, but its resulting accuracy highly depends on a
training/fine-tuning process and requires the original data. This not only
brings heavy computation and time costs but also is not conducive to privacy
and sensitive information protection. Therefore, a few recent works are
starting to focus on data-free quantization. However, data-free quantization
does not perform well while dealing with ultra-low precision quantization.
Although researchers utilize generative methods of synthetic data to address
this problem partially, data synthesis needs to take a lot of computation and
time. In this paper, we propose a data-free mixed-precision compensation
(DF-MPC) method to recover the performance of an ultra-low precision quantized
model without any data and fine-tuning process. By assuming the quantized error
caused by a low-precision quantized layer can be restored via the
reconstruction of a high-precision quantized layer, we mathematically formulate
the reconstruction loss between the pre-trained full-precision model and its
layer-wise mixed-precision quantized model. Based on our formulation, we
theoretically deduce the closed-form solution by minimizing the reconstruction
loss of the feature maps. Since DF-MPC does not require any original/synthetic
data, it is a more efficient method to approximate the full-precision model.
Experimentally, our DF-MPC is able to achieve higher accuracy for an ultra-low
precision quantized model compared to the recent methods without any data and
fine-tuning process.

本文提出了一种数据自由的混合精度补偿方法，通过数学公式对预训练的全精度模型和其逐层混合精度量化模型之间的重构损失进行最小化，无需任何数据和微调过程即可实现超低精度量化模型的更高精度。