Post-training quantization (PTQ) has been gaining popularity for the
deployment of deep neural networks on resource-limited devices since unlike
quantization-aware training, neither a full training dataset nor end-to-end
training is required at all. As PTQ schemes based on reconstructing each layer
or block output turn out to be effective to enhance quantized model
performance, recent works have developed algorithms to devise and learn a new
weight-rounding scheme so as to better reconstruct each layer or block output.
In this work, we propose a simple yet effective new weight-rounding mechanism
for PTQ, coined FlexRound, based on element-wise division instead of typical
element-wise addition such that FlexRound enables jointly learning a common
quantization grid size as well as a different scale for each pre-trained
weight. Thanks to the reciprocal rule of derivatives induced by element-wise
division, FlexRound is inherently able to exploit pre-trained weights when
updating their corresponding scales, and thus, flexibly quantize pre-trained
weights depending on their magnitudes. We empirically validate the efficacy of
FlexRound on a wide range of models and tasks. To the best of our knowledge,
our work is the first to carry out comprehensive experiments on not only image
classification and natural language understanding but also natural language
generation, assuming a per-tensor uniform PTQ setting. Moreover, we
demonstrate, for the first time, that large language models can be efficiently
quantized, with only a negligible impact on performance compared to
half-precision baselines, achieved by reconstructing the output in a
block-by-block manner.

本研究提出了一种名为 FlexRound 的新的权重取整机制，通过元素除法实现对预训练权重的灵活量化，可以一起学习一个公共的量化网格大小以及每个预训练权重的不同比例。实验结果表明，该机制用于后训练量化可以有效提高模型性能，并且还可以在图像分类、自然语言理解和生成等多个领域实现量化。