With the rising popularity of Large Language Models (LLMs), there has been an
increasing interest in compression techniques that enable their efficient
deployment. This study focuses on the Post-Training Quantization (PTQ) of LLMs.
Drawing from recent advances, our work introduces QuantEase, a layer-wise
quantization framework where individual layers undergo separate quantization.
The problem is framed as a discrete-structured non-convex optimization,
prompting the development of algorithms rooted in Coordinate Descent (CD)
techniques. These CD-based methods provide high-quality solutions to the
complex non-convex layer-wise quantization problems. Notably, our CD-based
approach features straightforward updates, relying solely on matrix and vector
operations, circumventing the need for matrix inversion or decomposition. We
also explore an outlier-aware variant of our approach, allowing for retaining
significant weights (outliers) with complete precision. Our proposal attains
state-of-the-art performance in terms of perplexity and zero-shot accuracy in
empirical evaluations across various LLMs and datasets, with relative
improvements up to 15% over methods such as GPTQ. Particularly noteworthy is
our outlier-aware algorithm's capability to achieve near or sub-3-bit
quantization of LLMs with an acceptable drop in accuracy, obviating the need
for non-uniform quantization or grouping techniques, improving upon methods
such as SpQR by up to two times in terms of perplexity.

本研究采用 Coordinate Descent（CD）技术，通过 QuantEase 分层量化框架对 Large Language Models（LLMs）进行后训练量化，包括离群值感知算法，以实现近乎 3 位量化并提高模型性能。