The ever-growing computational complexity of Large Language Models (LLMs)
necessitates efficient deployment strategies. The current state-of-the-art
approaches for Post-training Quantization (PTQ) often require calibration to
achieve the desired accuracy. This paper presents AdpQ, a novel zero-shot
adaptive PTQ method for LLMs that achieves the state-of-the-art performance in
low-precision quantization (e.g. 3-bit) without requiring any calibration data.
Inspired by Adaptive LASSO regression model, our proposed approach tackles the
challenge of outlier activations by separating salient weights using an
adaptive soft-thresholding method. Guided by Adaptive LASSO, this method
ensures that the quantized weights distribution closely follows the originally
trained weights and eliminates the need for calibration data entirely, setting
our method apart from popular approaches such as SpQR and AWQ. Furthermore, our
method offers an additional benefit in terms of privacy preservation by
eliminating any calibration or training data. We also delve deeper into the
information-theoretic underpinnings of the proposed method. We demonstrate that
it leverages the Adaptive LASSO to minimize the Kullback-Leibler divergence
between the quantized weights and the originally trained weights. This
minimization ensures the quantized model retains the Shannon information
content of the original model to a great extent, guaranteeing efficient
deployment without sacrificing accuracy or information. Our results achieve the
same accuracy as the existing methods on various LLM benchmarks while the
quantization time is reduced by at least 10x, solidifying our contribution to
efficient and privacy-preserving LLM deployment.

本研究提出了一种名为 AdpQ 的零样本自适应 PTQ 方法，通过采用自适应软阈值方法分离显著权重，实现低精度量化（如 3 位）中无需任何校准数据的最先进性能，进一步提供隐私保护优势，而且在各种 LLM 基准测试中达到与现有方法相同的准确性，即使量化时间缩短至少 10 倍。