This paper explores network binarization, a radical form of quantization,
compressing model weights to a single bit, specifically for Large Language
Models (LLMs) compression. Due to previous binarization methods collapsing
LLMs, we propose a novel approach, Partially-Binarized LLM (PB-LLM), which can
achieve extreme low-bit quantization while maintaining the linguistic reasoning
capacity of quantized LLMs. Specifically, our exploration first uncovers the
ineffectiveness of naive applications of existing binarization algorithms and
highlights the imperative role of salient weights in achieving low-bit
quantization. Thus, PB-LLM filters a small ratio of salient weights during
binarization, allocating them to higher-bit storage, i.e.,
partially-binarization. PB-LLM is extended to recover the capacities of
quantized LMMs, by analyzing from the perspective of post-training quantization
(PTQ) and quantization-aware training (QAT). Under PTQ, combining the concepts
from GPTQ, we reconstruct the binarized weight matrix guided by the Hessian
matrix and successfully recover the reasoning capacity of PB-LLM in low-bit.
Under QAT, we freeze the salient weights during training, explore the
derivation of optimal scaling factors crucial for minimizing the quantization
error, and propose a scaling mechanism based on this derived scaling strategy
for residual binarized weights. Those explorations and the developed
methodologies significantly contribute to rejuvenating the performance of
low-bit quantized LLMs and present substantial advancements in the field of
network binarization for LLMs.The code is available at
this https URL

本文研究了网络二值化技术在大型语言模型（LLMs）压缩中的应用。通过提出一种新的方法，即部分二值化 LLMs（PB-LLM），我们能够在保持低位量化的同时保持 LLMs 的语言推理能力。我们通过后训练量化（PTQ）和量化感知训练（QAT）的分析，提出了恢复量化 LLMs 容量的方法。这些研究和方法对于改善低位量化 LLMs 的性能以及网络二值化在 LLMs 领域的发展具有重要的意义。