Recently, post-training quantization (PTQ) has driven much attention to
produce efficient neural networks without long-time retraining. Despite its low
cost, current PTQ works tend to fail under the extremely low-bit setting. In
this study, we pioneeringly confirm that properly incorporating activation
quantization into the PTQ reconstruction benefits the final accuracy. To deeply
understand the inherent reason, a theoretical framework is established,
indicating that the flatness of the optimized low-bit model on calibration and
test data is crucial. Based on the conclusion, a simple yet effective approach
dubbed as QDROP is proposed, which randomly drops the quantization of
activations during PTQ. Extensive experiments on various tasks including
computer vision (image classification, object detection) and natural language
processing (text classification and question answering) prove its superiority.
With QDROP, the limit of PTQ is pushed to the 2-bit activation for the first
time and the accuracy boost can be up to 51.49%. Without bells and whistles,
QDROP establishes a new state of the art for PTQ. Our code is available at
this https URL and has been integrated into MQBench
(this https URL)

本文提出了一种新的方法，即将 activation quantization 纳入 post-training quantization 中，随机放弃激活量化，从而将 PTQ 的极限推向了 2 位。最终结果表明，QDROP 在图像分类、目标检测和文本分类任务中具有优异的表现，成为了 PTQ 现有技术中新的最佳方法。