Quantization of deep neural networks (DNN) has been proven effective for
compressing and accelerating DNN models. Data-free quantization (DFQ) is a
promising approach without the original datasets under privacy-sensitive and
confidential scenarios. However, current DFQ solutions degrade accuracy, need
synthetic data to calibrate networks, and are time-consuming and costly. This
paper proposes an on-the-fly DFQ framework with sub-second quantization time,
called SQuant, which can quantize networks on inference-only devices with low
computation and memory requirements. With the theoretical analysis of the
second-order information of DNN task loss, we decompose and approximate the
Hessian-based optimization objective into three diagonal sub-items, which have
different areas corresponding to three dimensions of weight tensor:
element-wise, kernel-wise, and output channel-wise. Then, we progressively
compose sub-items and propose a novel data-free optimization objective in the
discrete domain, minimizing Constrained Absolute Sum of Error (or CASE in
short), which surprisingly does not need any dataset and is even not aware of
network architecture. We also design an efficient algorithm without
back-propagation to further reduce the computation complexity of the objective
solver. Finally, without fine-tuning and synthetic datasets, SQuant accelerates
the data-free quantization process to a sub-second level with >30% accuracy
improvement over the existing data-free post-training quantization works, with
the evaluated models under 4-bit quantization. We have open-sourced the SQuant
framework at this https URL.

本篇论文提出了一种名为 SQuant 的实时数据无需模型压缩框架，采用三对角线子项目组合构成新颖的无数据最优化目标解算法，同时设计了一个有效的算法来进一步减少目标求解的计算复杂度。实验结果表明，SQuant 不需要微调和合成数据集，可以将数据无需量化过程加速到次秒级，并且在 4-bit 量化下，精度比现有数据无需压缩方法提高了 30% 以上