This paper investigates three different parameterizations of asymmetric
uniform quantization for quantization-aware training: (1) scale and offset, (2)
minimum and maximum, and (3) beta and gamma. We perform a comprehensive
comparative analysis of these parameterizations' influence on
quantization-aware training, using both controlled experiments and real-world
large language models. Our particular focus is on their changing behavior in
response to critical training hyperparameters, bit width and learning rate.
Based on our investigation, we propose best practices to stabilize and
accelerate quantization-aware training with learnable asymmetric quantization
ranges.

本研究通过比较分析三种不对称均匀量化的参数化方法，即比例和偏移、最小值和最大值以及 β 和 γ，来研究其对量化感知训练的影响。我们使用控制实验和实际大型语言模型的数据进行全面分析，并着重探讨其在关键训练超参数、比特宽度和学习率变化时的行为。根据我们的研究结果，我们提出了稳定和加速可学习不对称量化范围的量化感知训练的最佳实践。