Large language models (LLMs) have proven to be very superior to conventional
methods in various tasks. However, their expensive computations and high memory
requirements are prohibitive for deployment. Model quantization is an effective
method for reducing this overhead. The problem is that in most previous works,
the quantized model was calibrated using few samples from the training data,
which might affect the generalization of the quantized LLMs to unknown cases
and tasks. Hence in this work, we explore an important question: Can we design
a data-independent quantization method for LLMs to guarantee its generalization
performance? In this work, we propose EasyQuant, a training-free and
data-independent weight-only quantization algorithm for LLMs. Our observation
indicates that two factors: outliers in the weight and quantization ranges, are
essential for reducing the quantization error. Therefore, in EasyQuant, we
leave the outliers (less than 1%) unchanged and optimize the quantization range
to reduce the reconstruction error. With these methods, we surprisingly find
that EasyQuant achieves comparable performance to the original model. Since
EasyQuant does not depend on any training data, the generalization performance
of quantized LLMs is safely guaranteed. Moreover, EasyQuant can be implemented
in parallel so that the quantized model could be attained in a few minutes even
for LLMs over 100B. To our best knowledge, we are the first work that achieves
almost lossless quantization performance for LLMs under a data-independent
setting and our algorithm runs over 10 times faster than the data-dependent
methods.

本研究提出了 EasyQuant，这是一种训练免费且独立于数据的权重量化算法，旨在实现对大型语言模型（LLMs）的几乎无损量化性能，且算法运行速度比依赖于数据的方法快 10 倍以上。

EasyQuant: 一种高效无数据量化算法用于 LLMs

EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs

Data-free quantization aims to achieve model quantization without accessing
any authentic sample. It is significant in an application-oriented context
involving data privacy. Converting noise vectors into synthetic samples through
a generator is a popular data-free quantization method, which is called
generative data-free quantization. However, there is a difference in attention
between synthetic samples and authentic samples. This is always ignored and
restricts the quantization performance. First, since synthetic samples of the
same class are prone to have homogenous attention, the quantized network can
only learn limited modes of attention. Second, synthetic samples in eval mode
and training mode exhibit different attention. Hence, the batch-normalization
statistics matching tends to be inaccurate. ACQ is proposed in this paper to
fix the attention of synthetic samples. An attention center position-condition
generator is established regarding the homogenization of intra-class attention.
Restricted by the attention center matching loss, the attention center position
is treated as the generator's condition input to guide synthetic samples in
obtaining diverse attention. Moreover, we design adversarial loss of paired
synthetic samples under the same condition to prevent the generator from paying
overmuch attention to the condition, which may result in mode collapse. To
improve the attention similarity of synthetic samples in different network
modes, we introduce a consistency penalty to guarantee accurate BN statistics
matching. The experimental results demonstrate that ACQ effectively improves
the attention problems of synthetic samples. Under various training settings,
ACQ achieves the best quantization performance. For the 4-bit quantization of
Resnet18 and Resnet50, ACQ reaches 67.55% and 72.23% accuracy, respectively.

该论文提出了 ACQ 方法，通过注意力中心位置条件发生器和对手损失等技术，解决了数据合成过程中 attention 分布不均和同一类别合成样本之间注意力过于相似的问题，以提升量化算法的性能。在对 Resnet18 和 Resnet50 进行 4 位量化时，ACQ 方法分别实现了 67.55% 和 72.23% 的准确率。