Large language models (LLMs) have recently demonstrated remarkable
performance across diverse language tasks. But their deployment is often
constrained by their substantial computational and storage requirements.
Quantization has emerged as a key technique for addressing this challenge,
enabling the compression of large models with minimal impact on performance.
The recent GPTQ algorithm, a post-training quantization (PTQ) method, has
proven highly effective for compressing LLMs, sparking a wave of research that
leverages GPTQ as a core component. Recognizing the pivotal role of GPTQ in the
PTQ landscape, we introduce CDQuant, a simple and scalable alternative to GPTQ
with improved performance. CDQuant uses coordinate descent to minimize the
layer-wise reconstruction loss to achieve high-quality quantized weights. Our
algorithm is easy to implement and scales efficiently to models with hundreds
of billions of parameters. Through extensive evaluation on the PaLM2 model
family, we demonstrate that CDQuant consistently outperforms GPTQ across
diverse model sizes and quantization levels. In particular, for INT2
quantization of PaLM2-Otter, CDQuant achieves a 10% reduction in perplexity
compared to GPTQ.

CDQuant 是一个简单且可扩展的替代 GPTQ 的算法，使用坐标下降法来实现高质量的量化权重，通过在 PaLM2 模型系列上进行广泛评估，我们证明 CDQuant 在各种模型规模和量化级别下始终优于 GPTQ。

CDQuant: 使用贪婪协调下降的准确大型预训练模型的训练后权重量化

CDQuant: Accurate Post-training Weight Quantization of Large Pre-trained  Models using Greedy Coordinate Descent

Quantization has become a crucial step for the efficient deployment of deep
neural networks, where floating point operations are converted to simpler fixed
point operations. In its most naive form, it simply consists in a combination
of scaling and rounding transformations, leading to either a limited
compression rate or a significant accuracy drop. Recently, Gradient-based
post-training quantization (GPTQ) methods appears to be constitute a suitable
trade-off between such simple methods and more powerful, yet expensive
Quantization-Aware Training (QAT) approaches, particularly when attempting to
quantize LLMs, where scalability of the quantization process is of paramount
importance. GPTQ essentially consists in learning the rounding operation using
a small calibration set. In this work, we challenge common choices in GPTQ
methods. In particular, we show that the process is, to a certain extent,
robust to a number of variables (weight selection, feature augmentation, choice
of calibration set). More importantly, we derive a number of best practices for
designing more efficient and scalable GPTQ methods, regarding the problem
formulation (loss, degrees of freedom, use of non-uniform quantization schemes)
or optimization process (choice of variable and optimizer). Lastly, we propose
a novel importance-based mixed-precision technique. Those guidelines lead to
significant performance improvements on all the tested state-of-the-art GPTQ
methods and networks (e.g. +6.819 points on ViT for 4-bit quantization), paving
the way for the design of scalable, yet effective quantization methods.

量化方法在深度神经网络的高效部署中变得至关重要，深度神经网络经常需要量化以便在计算中使用固定点操作代替浮点操作。本文探讨了一种基于梯度的后训练量化方法（GPTQ），证明了该方法在选择权重、特征增强、校准集等方面具有一定鲁棒性，并提出了设计更高效、可扩展的 GPTQ 方法的准则，最后还提出了一种基于重要性的混合精度技术，这些准则和技术共同促进了已有的 GPTQ 方法和网络的性能改进，为设计可扩展且有效的量化方法开辟了新的可能。