In this paper, we introduce an algorithm for data quantization based on the
principles of Kashin representation. This approach hinges on decomposing any
given vector, matrix, or tensor into two factors. The first factor maintains a
small infinity norm, while the second exhibits a similarly constrained norm
when multiplied by an orthogonal matrix. Surprisingly, the entries of factors
after decomposition are well-concentrated around several peaks, which allows us
to efficiently replace them with corresponding centroids for quantization
purposes. We study the theoretical properties of the proposed approach and
rigorously evaluate our compression algorithm in the context of next-word
prediction tasks and on a set of downstream tasks for text classification. Our
findings demonstrate that Kashin Quantization achieves competitive or superior
quality in model performance while ensuring data compression, marking a
significant advancement in the field of data quantization.

基于 Kashin 表示原理，我们引入了一种数据量化算法，通过将给定的向量、矩阵或张量分解为两个因子，其中第一个因子具有小的无穷范数，而第二个在乘以正交矩阵后也具有相似的限制范数。通过分解后的因子条目意外地集中在几个峰值附近，允许我们有效地通过相应的质心进行量化。我们研究了该方法的理论性质，并在下一个单词预测任务和一组文本分类的下游任务中对我们的压缩算法进行了严格评估。我们的研究结果表明，Kashin 量化在保证数据压缩的同时，实现了竞争性或更高的模型性能质量，标志着数据量化领域的重要进展。

超定基的大型语言模型的量化

Quantization of Large Language Models with an Overdetermined Basis

Data quantization is an effective method to accelerate neural network
training and reduce power consumption. However, it is challenging to perform
low-bit quantized training: the conventional equal-precision quantization will
lead to either high accuracy loss or limited bit-width reduction, while
existing mixed-precision methods offer high compression potential but failed to
perform accurate and efficient bit-width assignment. In this work, we propose
DYNASTY, a block-wise dynamic-precision neural network training framework.
DYNASTY provides accurate data sensitivity information through fast online
analytics, and maintains stable training convergence with an adaptive bit-width
map generator. Network training experiments on CIFAR-100 and ImageNet dataset
are carried out, and compared to 8-bit quantization baseline, DYNASTY brings up
to $5.1\times$ speedup and $4.7\times$ energy consumption reduction with no
accuracy drop and negligible hardware overhead.

提出了一种动态精度神经网络训练框架 DYNASTY，通过快速在线分析提供准确的数据敏感度信息并通过自适应位宽映射生成器维持稳定的训练收敛性，与传统 8 位量化相比，DYNASTY 在 CIFAR-100 和 ImageNet 数据集上带来高达 5.1 倍的加速和 4.7 倍的能耗降低，无误差降低及可忽略的硬件开销。