Quantization of deep neural networks (DNN) has become a key element in the
efforts of embedding such networks on end-user devices. However, current
quantization methods usually suffer from costly accuracy degradation. In this
paper, we propose a new method for Enhanced Post Training Quantization named
EPTQ. The method is based on knowledge distillation with an adaptive weighting
of layers. In addition, we introduce a new label-free technique for
approximating the Hessian trace of the task loss, named Label-Free Hessian.
This technique removes the requirement of a labeled dataset for computing the
Hessian. The adaptive knowledge distillation uses the Label-Free Hessian
technique to give greater attention to the sensitive parts of the model while
performing the optimization. Empirically, by employing EPTQ we achieve
state-of-the-art results on a wide variety of models, tasks, and datasets,
including ImageNet classification, COCO object detection, and Pascal-VOC for
semantic segmentation. We demonstrate the performance and compatibility of EPTQ
on an extended set of architectures, including CNNs, Transformers, hybrid, and
MLP-only models.

本研究提出了一种增强后训练量化方法，名为 EPTQ，该方法基于层的自适应加权进行知识蒸馏。此外，我们引入了一种无标签技术用于估计任务损失的 Hessian 迹，名为 Label-Free Hessian。通过采用 EPTQ，我们在各种模型、任务和数据集上取得了最先进的结果，包括 ImageNet 分类、COCO 目标检测和 Pascal-VOC 语义分割。我们还展示了 EPTQ 在各种体系结构上的性能和兼容性，包括 CNN、Transformer、混合和仅 MLP 模型。