Lately, post-training quantization methods have gained considerable
attention, as they are simple to use, and require only a small unlabeled
calibration set. This small dataset cannot be used to fine-tune the model
without significant over-fitting. Instead, these methods only use the
calibration set to set the activations' dynamic ranges. However, such methods
always resulted in significant accuracy degradation, when used below 8-bits
(except on small datasets). Here we aim to break the 8-bit barrier. To this
end, we minimize the quantization errors of each layer separately by optimizing
its parameters over the calibration set. We empirically demonstrate that this
approach is: (1) much less susceptible to over-fitting than the standard
fine-tuning approaches, and can be used even on a very small calibration set;
and (2) more powerful than previous methods, which only set the activations'
dynamic ranges. Furthermore, we demonstrate how to optimally allocate the
bit-widths for each layer, while constraining accuracy degradation or model
compression by proposing a novel integer programming formulation. Finally, we
suggest model global statistics tuning, to correct biases introduced during
quantization. Together, these methods yield state-of-the-art results for both
vision and text models. For instance, on ResNet50, we obtain less than 1\%
accuracy degradation --- with 4-bit weights and activations in all layers, but
the smallest two. We open-sourced our code.

本文研究使用后训练量化方法达到低于 8 位的精度损失和模型压缩，采用分层次优化参数和整数规划方案，性能在视觉和文本模型上最优。