Model quantification uses low bit-width values to represent the weight
matrices of models, which is a promising approach to reduce both storage and
computational overheads of deploying highly anticipated LLMs. However, existing
quantization methods suffer severe performance degradation when the bit-width
is extremely reduced, and thus focus on utilizing 4-bit or 8-bit values to
quantize models. This paper boldly quantizes the weight matrices of LLMs to
1-bit, paving the way for the extremely low bit-width deployment of LLMs. For
this target, we introduce a 1-bit quantization-aware training (QAT) framework
named OneBit, including a novel 1-bit parameter representation method to better
quantize LLMs as well as an effective parameter initialization method based on
matrix decomposition to improve the convergence speed of the QAT framework.
Sufficient experimental results indicate that OneBit achieves good performance
(at least 83% of the non-quantized performance) with robust training processes
when only using 1-bit weight matrices.

该研究使用 1 位量化来减少高度期望的低精度模型的存储和计算开销，并通过引入一种 1 位量化感知训练框架 OneBit 以及基于矩阵分解的参数初始化方法来实现良好的性能（至少达到非量化性能的 83%）。

OneBit: 极低位大型语言模型

OneBit: Towards Extremely Low-bit Large Language Models

Deep neural network (DNN) quantization converting floating-point (FP) data in
the network to integers (INT) is an effective way to shrink the model size for
memory saving and simplify the operations for compute acceleration. Recently,
researches on DNN quantization develop from inference to training, laying a
foundation for the online training on accelerators. However, existing schemes
leaving batch normalization (BN) untouched during training are mostly
incomplete quantization that still adopts high precision FP in some parts of
the data paths. Currently, there is no solution that can use only low bit-width
INT data during the whole training process of large-scale DNNs with acceptable
accuracy. In this work, through decomposing all the computation steps in DNNs
and fusing three special quantization functions to satisfy the different
precision requirements, we propose a unified complete quantization framework
termed as ``WAGEUBN'' to quantize DNNs involving all data paths including W
(Weights), A (Activation), G (Gradient), E (Error), U (Update), and BN.
Moreover, the Momentum optimizer is also quantized to realize a completely
quantized framework. Experiments on ResNet18/34/50 models demonstrate that
WAGEUBN can achieve competitive accuracy on the ImageNet dataset. For the first
time, the study of quantization in large-scale DNNs is advanced to the full
8-bit INT level. In this way, all the operations in the training and inference
can be bit-wise operations, pushing towards faster processing speed, decreased
memory cost, and higher energy efficiency. Our throughout quantization
framework has great potential for future efficient portable devices with online
learning ability.

本文提出了一种名为 WAGEUBN 的统一完整量化框架，可将神经网络的数据路径（包括所有数据类型 W，A，G，E，U 和 BN）中所有数据转换为低位整数以实现全面量化和在线训练，并在 ImageNet 数据集上获得了可比较的精度，展示了在大规模 DNNs 中 8 位 INT 级别的完整量化的功能性。