The emergence of accurate open large language models (LLMs) has led to a race
towards quantization techniques for such models enabling execution on end-user
devices. In this paper, we revisit the problem of "extreme" LLM
compression--defined as targeting extremely low bit counts, such as 2 to 3 bits
per parameter, from the point of view of classic methods in Multi-Codebook
Quantization (MCQ). Our work builds on top of Additive Quantization, a classic
algorithm from the MCQ family, and adapts it to the quantization of language
models. The resulting algorithm advances the state-of-the-art in LLM
compression, outperforming all recently-proposed techniques in terms of
accuracy at a given compression budget. For instance, when compressing Llama 2
models to 2 bits per parameter, our algorithm quantizes the 7B model to 6.93
perplexity (a 1.29 improvement relative to the best prior work, and 1.81 points
from FP16), the 13B model to 5.70 perplexity (a .36 improvement) and the 70B
model to 3.94 perplexity (a .22 improvement) on WikiText2. We release our
implementation of Additive Quantization for Language Models AQLM as a baseline
to facilitate future research in LLM quantization.

该研究中，我们将经典的多码本量化方法应用于语言模型的压缩，实现了对开放式大规模语言模型的极端压缩，提高了在给定压缩预算下的准确性。